[issue43743] BlockingIOError: [Errno 11] Resource temporarily unavailable: on GPFS.

[Marvin Poul]

Marvin Poul <ponder at creshal.de> added the comment:

I hope you don't mind me necro posting, but I ran into this issue again and
have a small patch to solve it.

I attached an MWE that triggers the BlockingIOError reliably on ext4
filesystems in linux 4.12.14 and python 3.8.12.  Running under strace -e
sendfile gives the following output

# manually calling sendfile to check that it works
> sendfile(5, 4, [0] => [8388608], 8388608) = 8388608
# sendfile calls originating in shutil.copy
> sendfile(5, 4, [0] => [8388608], 8388608) = 8388608
> sendfile(5, 4, [8388608], 8388608)      = -1 EAGAIN (Resource temporarily unavailable)
> Shutil Failed!
> [Errno 11] Resource temporarily unavailable: '/cmmc/u/zora/scratch/sendfile_bug/tmpaqx2o4uj' -> '/cmmc/u/zora/scratch/sendfile_bug/tmpb8rzg8rg'
> +++ exited with 0 +++

This shows that the first call to sendfile actually copies the whole file and
the EAGAIN is only triggered on the second, unnecessary, call.  I have tested
with a small C program that it's triggered whenever sendfile's offset + count
exceeds the file size of in_fd.  This is weird behaviour on the kernels side
that seems to have changed in newer kernel versions (issue is not present e.g.
on my 5.16.12 laptop).

Anyways my patch makes that second call not appear by keeping track of the file
size and the bytes written so far.  It's against the current python main
branch, but if I see correctly this part hasn't changed in years.  I have
checked the error is not thrown when the patch is applied.

(I can only attach one file, so patch is attached in a new one.)

----------
nosy: +pmrv
Added file: https://bugs.python.org/file50671/sendfile.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43743>
_______________________________________