[New-bugs-announce] [issue35882] distutils fails with UnicodeEncodeError with strange filename in package_data

[Jody McIntyre]

New submission from Jody McIntyre <scjody at modernduck.com>:

I encountered an error while installing savReaderWriter using pip from a Dockerfile, reported as https://bitbucket.org/fomcl/savreaderwriter/issues/73/pip-install-fails-for-non-utf-8-encoding

This is actually an issue with distutils.

Steps to reproduce:

1. Create the following tree. setup.py is as attached. The .sav file is empty.

./setup.py
./savReaderWriter
./savReaderWriter/test_data
./savReaderWriter/test_data/scheiß Encoding.sav

2. Run LANG=C python setup.py install --record=/tmp/install-record.txt --single-version-externally-managed

Actual output:

running install
running build
running build_py
package init file 'savReaderWriter/__init__.py' not found (or not a regular file)
creating build
creating build/lib
creating build/lib/savReaderWriter
creating build/lib/savReaderWriter/test_data
copying savReaderWriter/test_data/scheiß Encoding.sav -> build/lib/savReaderWriter/test_data
running install_lib
running install_egg_info
running egg_info
creating savReaderWriter.egg-info
writing savReaderWriter.egg-info/PKG-INFO
writing dependency_links to savReaderWriter.egg-info/dependency_links.txt
writing top-level names to savReaderWriter.egg-info/top_level.txt
writing manifest file 'savReaderWriter.egg-info/SOURCES.txt'
'savReaderWriter/test_data/schei\udcc3\udc9f Encoding.sav' not utf-8 encodable -- skipping
reading manifest file 'savReaderWriter.egg-info/SOURCES.txt'
writing manifest file 'savReaderWriter.egg-info/SOURCES.txt'
removing '/home/scjody/.pyenv/versions/3.6.5/lib/python3.6/site-packages/savReaderWriter-1.2.3-py3.6.egg-info' (and everything under it)
Copying savReaderWriter.egg-info to /home/scjody/.pyenv/versions/3.6.5/lib/python3.6/site-packages/savReaderWriter-1.2.3-py3.6.egg-info
running install_scripts
writing list of installed files to '/tmp/install-record.txt'
Traceback (most recent call last):
  File "setup.py", line 9, in <module>
    version='1.2.3',
  File "/home/scjody/.pyenv/versions/3.6.5/lib/python3.6/site-packages/setuptools/__init__.py", line 129, in setup
    return distutils.core.setup(**attrs)
  File "/home/scjody/.pyenv/versions/3.6.5/lib/python3.6/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/home/scjody/.pyenv/versions/3.6.5/lib/python3.6/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/home/scjody/.pyenv/versions/3.6.5/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/home/scjody/.pyenv/versions/3.6.5/lib/python3.6/site-packages/setuptools/command/install.py", line 61, in run
    return orig.install.run(self)
  File "/home/scjody/.pyenv/versions/3.6.5/lib/python3.6/distutils/command/install.py", line 572, in run
    self.record)
  File "/home/scjody/.pyenv/versions/3.6.5/lib/python3.6/distutils/cmd.py", line 335, in execute
    util.execute(func, args, msg, dry_run=self.dry_run)
  File "/home/scjody/.pyenv/versions/3.6.5/lib/python3.6/distutils/util.py", line 301, in execute
    func(*args)
  File "/home/scjody/.pyenv/versions/3.6.5/lib/python3.6/distutils/file_util.py", line 236, in write_file
    f.write(line + "\n")
UnicodeEncodeError: 'ascii' codec can't encode characters in position 94-95: ordinal not in range(128)

Expected results:

The package is installed and install-record.txt is created.


Related: https://bugs.python.org/issue9561

----------
components: Distutils
files: setup.py
messages: 334687
nosy: dstufft, eric.araujo, scjody
priority: normal
severity: normal
status: open
title: distutils fails with UnicodeEncodeError with strange filename in package_data
type: crash
versions: Python 3.6
Added file: https://bugs.python.org/file48095/setup.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue35882>
_______________________________________