[issue36998] distutils sdist command fails to create MANIFEST if any filenames are undecodable

[Toshio Kuratomi]

Toshio Kuratomi <a.badger at gmail.com> added the comment:

Uploading a minimal test case.

$ tar -xzvf test-case.tar.gz
$ python3.7 setup.py sdist
running sdist
running check
warning: sdist: standard file not found: should have one of README, README.txt, README.rst

reading manifest template 'MANIFEST.in'
writing manifest file 'MANIFEST'
Traceback (most recent call last):
  File "setup.py", line 27, in <module>
    packages=['hello'],
  File "/usr/lib64/python3.7/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/lib64/python3.7/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/usr/lib64/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/usr/lib64/python3.7/distutils/command/sdist.py", line 152, in run
    self.get_file_list()
  File "/usr/lib64/python3.7/distutils/command/sdist.py", line 208, in get_file_list
    self.write_manifest()
  File "/usr/lib64/python3.7/distutils/command/sdist.py", line 390, in write_manifest
    "writing manifest file '%s'" % self.manifest)
  File "/usr/lib64/python3.7/distutils/cmd.py", line 335, in execute
    util.execute(func, args, msg, dry_run=self.dry_run)
  File "/usr/lib64/python3.7/distutils/util.py", line 291, in execute
    func(*args)
  File "/usr/lib64/python3.7/distutils/file_util.py", line 236, in write_file
    f.write(line + "\n")
UnicodeEncodeError: 'utf-8' codec can't encode character '\udcff' in position 11: surrogates not allowed

With PR applied:

$ ../../python setup.py sdist
running sdist
running check
warning: sdist: standard file not found: should have one of README, README.txt, README.rst

reading manifest template 'MANIFEST.in'
writing manifest file 'MANIFEST'
creating hello-1.0
creating hello-1.0/hello
creating hello-1.0/tests
creating hello-1.0/tests/data
making hard links in hello-1.0...
hard linking setup.py -> hello-1.0
hard linking hello/__init__.py -> hello-1.0/hello
hard linking tests/test_cases.py -> hello-1.0/tests
hard linking tests/data/1.bin -> hello-1.0/tests/data
hard linking tests/data/\udcff.bin -> hello-1.0/tests/data
Creating tar archive
removing 'hello-1.0' (and everything under it)

Making this minimal test case, though, I found that there's another error somewhere when MANIFEST has already been created (ie: the patched version works for the initial generation of MANIFEST but it doesn't work to *regenerate* the MANIFEST).  Looking into that now.

----------
Added file: https://bugs.python.org/file48349/test-case.tar.gz

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36998>
_______________________________________