[issue35772] test_tarfile fails on ppc64le when using tmpfs filesystem

[STINNER Victor]

New submission from STINNER Victor <vstinner at redhat.com>:

The following test_tarfile tests fail on ppc64 when using tmpfs filesystem (which is the case on RHEL package build server):

* test_sparse_file_00 (test.test_tarfile.GNUReadTest)
* test_sparse_file_01 (test.test_tarfile.GNUReadTest)
* test_sparse_file_10 (test.test_tarfile.GNUReadTest)
* test_sparse_file_old (test.test_tarfile.GNUReadTest)

Example of failure:

======================================================================
FAIL: test_sparse_file_00 (test.test_tarfile.GNUReadTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/Python-3.6.6/Lib/test/test_tarfile.py", line 964, in test_sparse_file_00
    self._test_sparse_file("gnu/sparse-0.0")
  File "/builddir/build/BUILD/Python-3.6.6/Lib/test/test_tarfile.py", line 958, in _test_sparse_file
    self.assertLess(s.st_blocks * 512, s.st_size)
AssertionError: 131072 not less than 86016

Bug first report on RHEL8:
https://bugzilla.redhat.com/show_bug.cgi?id=1639490


test_tarfile has _fs_supports_holes() function to check if the filesystem supports sparse files with holes. The function returns True on:

* ext4 filesystem on x86_64 on my Fedora 29 (kernel 4.19)
* ext4 filesystem on x86_64 on my Fedora 29 (kernel 4.19)
* XFS filesystem on ppc64le (kernel 4.18)
* tmpfs filesystem on ppc64le (kernel 4.18)

In short, it always return True on x86_64 and ppc64le Linux kernels.

Problem: in practice, "tmpfs filesystem on ppc64le (kernel 4.18)" doesn't fully support sparse files.

--

Example from:
https://bugzilla.redhat.com/show_bug.cgi?id=1639490#c5

# ls -lhs ~/sparse 
48K -rw-r--r--. 1 root root 84K Jan 18 05:36 /root/sparse

Copy a sparse file from XFS to tmpfs: cp --sparse=always and fallocate --dig fail to punch holes, the file always take 128K on disk on tmpfs.

# cp sparse /root/mytmp/sparse --sparse=always
# ls -lhs /root/mytmp/sparse 
128K -rw-r--r--. 1 root root 84K Jan 18 06:10 /root/mytmp/sparse
# fallocate --dig /root/mytmp/sparse
# ls -lhs /root/mytmp/sparse 
128K -rw-r--r--. 1 root root 84K Jan 18 06:10 /root/mytmp/sparse

Counter example on XFS, source and destionation files use 48K on disk fo 84K of data:

# cp sparse sparse2 --sparse=always
# ls -lhs sparse*
48K -rw-r--r--. 1 root root 84K Jan 18 05:36 sparse
48K -rw-r--r--. 1 root root 84K Jan 18 06:13 sparse2

--

Attached PR fix the _fs_support_holes() heuristic to return properly False on tmpfs on ppc64le.

----------
components: Tests
messages: 333956
nosy: vstinner
priority: normal
severity: normal
status: open
title: test_tarfile fails on ppc64le when using tmpfs filesystem
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue35772>
_______________________________________