[Borgbackup] Version 1.0.3 has grown a new failing test in test_sparse_file

[Hans-Peter Jansen]

Hi,

while preparing 1.0.3 for openSUSE, I noticed two more failing tests:

[  112s] =================================== FAILURES ===================================
[  112s] ______________________ ArchiverTestCase.test_sparse_file _______________________
[  112s] 
[  112s] self = <borg.testsuite.archiver.ArchiverTestCase testMethod=test_sparse_file>
[  112s] 
[  112s]     def test_sparse_file(self):
[  112s]         # no sparse file support on Mac OS X
[  112s]         sparse_support = sys.platform != 'darwin'
[  112s]         filename = os.path.join(self.input_path, 'sparse')
[  112s]         content = b'foobar'
[  112s]         hole_size = 5 * (1 << CHUNK_MAX_EXP)  # 5 full chunker buffers
[  112s]         with open(filename, 'wb') as fd:
[  112s]             # create a file that has a hole at the beginning and end (if the
[  112s]             # OS and filesystem supports sparse files)
[  112s]             fd.seek(hole_size, 1)
[  112s]             fd.write(content)
[  112s]             fd.seek(hole_size, 1)
[  112s]             pos = fd.tell()
[  112s]             fd.truncate(pos)
[  112s]         total_len = hole_size + len(content) + hole_size
[  112s]         st = os.stat(filename)
[  112s]         self.assert_equal(st.st_size, total_len)
[  112s]         if sparse_support and hasattr(st, 'st_blocks'):
[  112s]             self.assert_true(st.st_blocks * 512 < total_len / 9)  # is input sparse?
[  112s]         self.cmd('init', self.repository_location)
[  112s]         self.cmd('create', self.repository_location + '::test', 'input')
[  112s]         with changedir('output'):
[  112s]             self.cmd('extract', '--sparse', self.repository_location + '::test')
[  112s]         self.assert_dirs_equal('input', 'output/input')
[  112s]         filename = os.path.join(self.output_path, 'input', 'sparse')
[  112s]         with open(filename, 'rb') as fd:
[  112s]             # check if file contents are as expected
[  112s]             self.assert_equal(fd.read(hole_size), b'\0' * hole_size)
[  112s]             self.assert_equal(fd.read(len(content)), content)
[  112s]             self.assert_equal(fd.read(hole_size), b'\0' * hole_size)
[  112s]         st = os.stat(filename)
[  112s]         self.assert_equal(st.st_size, total_len)
[  112s]         if sparse_support and hasattr(st, 'st_blocks'):
[  112s] >           self.assert_true(st.st_blocks * 512 < total_len / 9)  # is output sparse?
[  112s] E           AssertionError: False is not true
[  112s] 
[  112s] borg/testsuite/archiver.py:416: AssertionError
[  112s] ___________________ RemoteArchiverTestCase.test_sparse_file ____________________
[  112s] 
[  112s] self = <borg.testsuite.archiver.RemoteArchiverTestCase testMethod=test_sparse_file>
[  112s] 
[  112s]     def test_sparse_file(self):
[  112s]         # no sparse file support on Mac OS X
[  112s]         sparse_support = sys.platform != 'darwin'
[  112s]         filename = os.path.join(self.input_path, 'sparse')
[  112s]         content = b'foobar'
[  112s]         hole_size = 5 * (1 << CHUNK_MAX_EXP)  # 5 full chunker buffers
[  112s]         with open(filename, 'wb') as fd:
[  112s]             # create a file that has a hole at the beginning and end (if the
[  112s]             # OS and filesystem supports sparse files)
[  112s]             fd.seek(hole_size, 1)
[  112s]             fd.write(content)
[  112s]             fd.seek(hole_size, 1)
[  112s]             pos = fd.tell()
[  112s]             fd.truncate(pos)
[  112s]         total_len = hole_size + len(content) + hole_size
[  112s]         st = os.stat(filename)
[  112s]         self.assert_equal(st.st_size, total_len)
[  112s]         if sparse_support and hasattr(st, 'st_blocks'):
[  112s]             self.assert_true(st.st_blocks * 512 < total_len / 9)  # is input sparse?
[  112s]         self.cmd('init', self.repository_location)
[  112s]         self.cmd('create', self.repository_location + '::test', 'input')
[  112s]         with changedir('output'):
[  112s]             self.cmd('extract', '--sparse', self.repository_location + '::test')
[  112s]         self.assert_dirs_equal('input', 'output/input')
[  112s]         filename = os.path.join(self.output_path, 'input', 'sparse')
[  112s]         with open(filename, 'rb') as fd:
[  112s]             # check if file contents are as expected
[  112s]             self.assert_equal(fd.read(hole_size), b'\0' * hole_size)
[  112s]             self.assert_equal(fd.read(len(content)), content)
[  112s]             self.assert_equal(fd.read(hole_size), b'\0' * hole_size)
[  112s]         st = os.stat(filename)
[  112s]         self.assert_equal(st.st_size, total_len)
[  112s]         if sparse_support and hasattr(st, 'st_blocks'):
[  112s] >           self.assert_true(st.st_blocks * 512 < total_len / 9)  # is output sparse?
[  112s] E           AssertionError: False is not true
[  112s] 
[  112s] borg/testsuite/archiver.py:416: AssertionError
[  112s] =================== 49 tests deselected by '-knot benchmark' ===================
[  112s] = 2 failed, 415 passed, 57 skipped, 49 deselected, 2 xfailed in 94.72 seconds ==
[  112s] error: Bad exit status from /var/tmp/rpm-tmp.bH663H (%check)

This is with openSUSE 13.2 and Kernel 4.2.5 on a xfs filesystem.

While at it, I noticed, that the document generation also fails.
The usual build sequence for packaging is:

CFLAGS="%{optflags}" python3 setup.py build
make -C docs html man && rm docs/_build/html/.buildinfo

resulting in:

[   14s] /home/abuild/rpmbuild/BUILD/borgbackup-1.0.3/docs/api.rst:5: WARNING: autodoc: failed to import module 'borg.archiver'; the following exception was raised:
[   14s] Traceback (most recent call last):
[   14s]   File "/usr/lib/python3.4/site-packages/sphinx/ext/autodoc.py", line 507, in import_object
[   14s]     __import__(self.modname)
[   14s]   File "/home/abuild/rpmbuild/BUILD/borgbackup-1.0.3/borg/archiver.py", line 19, in <module>
[   14s]     from .helpers import Error, location_validator, archivename_validator, format_line, format_time, format_file_size, \
[   14s]   File "/home/abuild/rpmbuild/BUILD/borgbackup-1.0.3/borg/helpers.py", line 25, in <module>
[   14s]     from . import hashindex
[   14s] ImportError: cannot import name 'hashindex'
[   14s] /home/abuild/rpmbuild/BUILD/borgbackup-1.0.3/docs/api.rst:9: WARNING: autodoc: failed to import module 'borg.upgrader'; the following exception was raised:
[   14s] Traceback (most recent call last):
[   14s]   File "/usr/lib/python3.4/site-packages/sphinx/ext/autodoc.py", line 507, in import_object
[   14s]     __import__(self.modname)
[   14s]   File "/home/abuild/rpmbuild/BUILD/borgbackup-1.0.3/borg/upgrader.py", line 9, in <module>
[   14s]     from .helpers import get_keys_dir, get_cache_dir, ProgressIndicatorPercent
[   14s]   File "/home/abuild/rpmbuild/BUILD/borgbackup-1.0.3/borg/helpers.py", line 25, in <module>
[   14s]     from . import hashindex
[   14s] ImportError: cannot import name 'hashindex'
[   14s] /home/abuild/rpmbuild/BUILD/borgbackup-1.0.3/docs/api.rst:13: WARNING: autodoc: failed to import module 'borg.archive'; the following exception was raised:
[   14s] Traceback (most recent call last):

Any idea, how to transform this sequence to succeed?

Cheers,
Pete