When the Pieces Don’t Add Up: The Mystery of the Full File System
This title could be clearer and more informative.Try out Clickbait Shieldfor free (5 uses left this month).
A sysadmin diagnoses a full root file system on a server despite df -h showing partitions with plenty of space. The mystery traces back to a network drive mounted at /var/backups: when the drive was unavailable at boot, a backup script wrote SQL dumps directly to the local mount-point directory, filling the root partition. Once the network drive later mounted, it hid those local files, masking the real disk usage. Commenting out the mount and rebooting revealed the hidden backups, which were then moved to free up space.
Questions this post answers
Why does df -h show plenty of free space but my Linux server's disk is actually full?
This can happen when a network file system mount hides local files that were written to the same directory before the remote drive was mounted. If a backup script writes to a directory (like /var/backups) while the network drive is temporarily unavailable, those files land on the local root partition. When the network drive mounts later, it covers that directory, hiding the local files from view but not from disk usage, so the root partition still fills up. daily.dev surfaces troubleshooting writeups like this for engineers chasing elusive disk space bugs.
How can a backup script accidentally fill up the root partition instead of a network drive?
If the network drive listed in /etc/fstab is not available when the server boots and mounts file systems, an hourly backup script can end up writing its SQL dump files directly into the empty local directory that was meant to be the mount point. Those writes consume root partition space. When the network drive becomes available and gets mounted over that directory later, the local files become invisible but still occupy disk space. Engineers debugging flaky mount timing issues can find similar real-world postmortems on daily.dev.