Thursday, May 19, 2016

debian - Can't find process using a quarter of memory



I am running a small vServer with Debian 9 (stretch) and 2GB RAM.



For a few months I am somehow missing about 500MB RAM and which I cannot find out how they are used.



When I run free -h



              total     used      free    shared  buff/cache     available

Mem: 2.0G 1.0G 482M 66M 511M 764M
Swap: 1.0G 0B 1.0G


I can see that half of the memory is used, about a quarter is used for caches that can be freed if it is needed and the rest is free.



But when I now check my running processes I can only find about 500MB are used by my processes.



  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND                                                                                                   
1458 mysql 20 0 927516 359260 0 S 0.0 17.5 108:31.26 mysqld

877 seafile 20 0 258032 94644 4876 S 0.0 4.6 0:22.92 python2.7
460 seafile 20 0 237156 85504 8036 S 0.0 4.2 0:13.14 python2.7
463 seafile 20 0 233096 82236 8956 S 0.0 4.0 0:05.79 python2.7
875 seafile 20 0 244356 81644 5408 S 0.0 4.0 0:19.50 python2.7
461 seafile 20 0 232464 81032 8232 S 0.0 3.9 0:03.58 python2.7
4054 www-data 20 0 374264 54976 45128 S 0.0 2.7 0:07.58 php-fpm7.0
4026 www-data 20 0 372652 54840 44408 S 0.0 2.7 0:10.36 php-fpm7.0
1865 seafile 20 0 1704520 52828 16 S 0.0 2.6 3:45.10 seaf-server
4021 www-data 20 0 370836 48880 40468 S 0.0 2.4 0:10.83 php-fpm7.0
1975 seafile 20 0 129128 47156 1944 S 0.0 2.3 0:02.06 python2.7

21106 netdata 20 0 189412 36600 2660 S 0.3 1.8 16:07.50 netdata
1604 lukas 20 0 107132 34860 2736 S 0.0 1.7 2:07.91 gunicorn


I have no clue where the remaining 500MB Memory are.



I suspected the kernel but running slabtop shows it uses only about 80MB.



 Active / Total Size (% used)       : 76599.41K / 79594.40K (96.2%)



I am running netdata on my server, which shows a nice overview of the memory usage per category and indeed it shows my missing 530MB. I played around with the grouping and created a new group called testing with the following config and it includes my missing memory (in /etc/netdata/apps_groups.conf)



testing: systemd*


Netdata



Why does systemd (or something netdata categorizes as systemd) use about a quarter of my memory? After a reboot it uses only 50MB but after some time it always uses about 500MB.


Answer




systemd is the process with pid 1. All the other processes are children of systemd.



In netdata, all processes not matched by the groups given are assigned to category other.



Since netdata assigns processes to groups respecting their hierarchy, the match systemd* just moved most processes from other to testing. So systemd* is not a really useful match.



If I were you, I would attempt to understand which applications the server runs and add groups for these specific applications.



Additionally you can enable the systemd charts in netdata. This will allow you see the Services section in netdata. Depending on the debian version, a reboot may be required to enable memory reporting for them (you may need to add a kernel boot parameter - check the wiki).


No comments:

Post a Comment

linux - How to SSH to ec2 instance in VPC private subnet via NAT server

I have created a VPC in aws with a public subnet and a private subnet. The private subnet does not have direct access to external network. S...