Sunday, December 4, 2016

Ubuntu "No space left on device" for /home, df shows 100% full, ds shows much, much less








On an Ubuntu 12.04 server, normal users can no longer create or add to files in /home, encountering a "No space left on device" error.



The /home directory has a capacity of 1.7 terabytes and as far as I can tell is nowhere near full in terms of actual data stored or inodes used.



df -h shows:



Filesystem      Size  Used Avail Use% Mounted on

/dev/md2 1.0T 18G 955G 2% /
udev 7.7G 4.0K 7.7G 1% /dev
tmpfs 3.1G 320K 3.1G 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 7.7G 0 7.7G 0% /run/shm
cgroup 7.7G 0 7.7G 0% /sys/fs/cgroup
/dev/md3 1.7T 1.7T 0 100% /home
/dev/md1 496M 45M 426M 10% /boot



/home indeed looks rather full.



du -hs /home suggests otherwise:



1.4G    /home


There appears no inode issue - df -i:



Filesystem        Inodes  IUsed     IFree IUse% Mounted on

/dev/md2 67108864 75334 67033530 1% /
udev 2013497 527 2012970 1% /dev
tmpfs 2015816 440 2015376 1% /run
none 2015816 2 2015814 1% /run/lock
none 2015816 1 2015815 1% /run/shm
cgroup 2015816 9 2015807 1% /sys/fs/cgroup
/dev/md3 113909760 105981 113803779 1% /home
/dev/md1 131072 239 130833 1% /boot



I recently deleted a many gigabytes of application cache and log data from /home, however this was in the tens of gigabytes at best and nowhere near the capcity of /home.



Update 1:



du -hs --apparent-size /home
1.2G /home
du -hs /home
1.4G /home



What might be going on here?

No comments:

Post a Comment

linux - How to SSH to ec2 instance in VPC private subnet via NAT server

I have created a VPC in aws with a public subnet and a private subnet. The private subnet does not have direct access to external network. S...