Friday, December 5, 2014

kernel - Mysql locks up linux server during high load

Every day during the generation of some reports the mysql process takes up 1 or 2 cores (100%). The problem is that the whole server becomes unresponsive during that time. I can't ssh, postfix doesn't work, apache neither. I can however ping the server. Cronjobs also work, so I'm currently saving top's output every other minute to a file.




There are more cores available, the memory usage is not that high so I don't know why this happens. There is not much I can do about the application that does the reports, but I find it very strange that everything else stops working when mysql is overloaded. Load average goes to around 1,5. After everything is over things start to behave normally. Is there anything I can do about it?



Ubuntu 12.04 with kernel 3.13 on a physical server.



top - 21:51:02 up 82 days, 13:57,  8 users,  load average: 1.19, 1.21, 1.15
Tasks: 367 total, 1 running, 364 sleeping, 0 stopped, 2 zombie
Cpu(s): 1.2%us, 0.9%sy, 1.4%ni, 95.7%id, 0.8%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 16274920k total, 13190728k used, 3084192k free, 284272k buffers
Swap: 4095996k total, 1006148k used, 3089848k free, 4570996k cached


PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
20794 mysql 20 0 3904m 893m 5260 S 100 5.6 2810:00 mysqld
19807 otrs 20 0 211m 46m 5560 S 75 0.3 0:00.39 otrs.PostMaster
14 root 20 0 0 0 0 S 2 0.0 20:49.52 rcuos/6

No comments:

Post a Comment

linux - How to SSH to ec2 instance in VPC private subnet via NAT server

I have created a VPC in aws with a public subnet and a private subnet. The private subnet does not have direct access to external network. S...