[issue42351] Setup.py: UnicodeDecodeError in grep_headers_for

[E. Paine]

New submission from E. Paine <paineelisha at gmail.com>:

When compiling the master branch (i.e. running 'make'), I get a UnicodeDecodeError as follows:
Traceback (most recent call last):
  File "/home/elisha/Documents/Python/cp0/cpython/./setup.py", line 2619, in <module>
    main()
  File "/home/elisha/Documents/Python/cp0/cpython/./setup.py", line 2589, in main
    setup(# PyPI Metadata (PEP 301)
  File "/home/elisha/Documents/Python/cp0/cpython/Lib/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/home/elisha/Documents/Python/cp0/cpython/Lib/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/home/elisha/Documents/Python/cp0/cpython/Lib/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/home/elisha/Documents/Python/cp0/cpython/Lib/distutils/command/build.py", line 135, in run
    self.run_command(cmd_name)
  File "/home/elisha/Documents/Python/cp0/cpython/Lib/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/home/elisha/Documents/Python/cp0/cpython/Lib/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/home/elisha/Documents/Python/cp0/cpython/Lib/distutils/command/build_ext.py", line 340, in run
    self.build_extensions()
  File "/home/elisha/Documents/Python/cp0/cpython/./setup.py", line 471, in build_extensions
    self.detect_modules()
  File "/home/elisha/Documents/Python/cp0/cpython/./setup.py", line 1825, in detect_modules
    self.detect_ctypes()
  File "/home/elisha/Documents/Python/cp0/cpython/./setup.py", line 2205, in detect_ctypes
    if grep_headers_for('ffi_prep_cif_var', ffi_headers):
  File "/home/elisha/Documents/Python/cp0/cpython/./setup.py", line 246, in grep_headers_for
    if function in f.read():
  File "/home/elisha/Documents/Python/cp0/cpython/Lib/codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x92 in position 4210: invalid start byte

The problematic file it is trying to read is /usr/include/OMX_Other.h which is part of the libomxil-bellagio package (a copy of this package can be downloaded from https://www.archlinux.org/packages/extra/x86_64/libomxil-bellagio/download/). More specifically, there are several characters in the comments which cannot be decoded correctly (the first of these is on line 93).

The fix is a very simple one and is just to add errors='replace' to line 244 of setup.py (I cannot see this having any ill-effects).

I couldn't find who to nosy for this so apologies about that.

----------
components: Build
messages: 380913
nosy: epaine
priority: normal
severity: normal
status: open
title: Setup.py: UnicodeDecodeError in grep_headers_for
type: compile error
versions: Python 3.10, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42351>
_______________________________________