[New-bugs-announce] [issue43638] MacOS os.statvfs() has rollover for >4TB disks at each 4TB (32bit counter overflow?)

[Sander]

New submission from Sander <sander.jonkers+github at gmail.com>:

MacOS BigSur (and older), python 3.9.2 (and older)

For disks >4TB, os.statvfs() shows a wrong value for available space: too low, and always rollover at each 4TB. 
As 4TB = 2^42, hypothesis: rollover in 32bit counter (with 10bit blocksize)

Example:

"df -m" does show the correct available space

df -m /Volumes/Frank/
Filesystem                                    1M-blocks    Used Available Capacity    iused       ifree %iused  Mounted on
//frank at SynologyBlabla._smb._tcp.local/Frank  21963360 2527744  19435615    12% 2588410474 19902070164   12%   /Volumes/Frank

So available space 19902070164 MB, so about 18.5 TB. Good.

Now python's os.statvfs():

>>> s = os.statvfs("/Volumes/Frank")
>>> s.f_bavail * s.f_frsize / 1024**2
2658399.39453125

So 2.5TB, and thus wrong

The difference is 16777216 MB which is exactly 4 times 4TB.

Problem seems to be in MacOS statvfs() itself; reproducable with a few lines of C code.

We have implemented a workaround in our python program SABnzbd to directly use MacOS' libc statfs() call (not statvfs() ). A solution / workaround in python itself would be much nicer.


No problem with python on Linux with >4TB drives.

----------
components: macOS
messages: 389596
nosy: ned.deily, ronaldoussoren, sanderjo
priority: normal
severity: normal
status: open
title: MacOS os.statvfs() has rollover for >4TB disks at each 4TB (32bit counter overflow?)
type: behavior
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43638>
_______________________________________