Monday, April 27, 2015

performance - How to make `rm` faster on ext3/linux?



I have ext3 filesystem mounted with default options. On it I have some ~ 100GB files.



Removal of any of such files takes long time (8 minutes) and causes a lot of io traffic, which increases load on server.



Is there any way to make the rm not as disruptive?


Answer




The most interesting answer was originally buried in a comment on the question. Here it is as a first class answer to make it more visible:




Basically no method from here worked, so we developed our own. Described it in here:
http://www.depesz.com/index.php/2010/04/04/how-to-remove-backups/ – depesz Apr 6 '10 at 15:15




That link is an incredibly thorough analysis of the exploration for and discovery of a workable solution.



Note also:




The article says:




As you can see, I used -c2 -n7 options to ionice, which seem sane.




which is true, but user TafT says if you want no disruption then -c3 'idle' would be a better choice than -c2 'best-effort'. He has used -c3 to build in the background and has found it to work well without causing the build to wait for ever. If you really do have 100% io usage then -c3 will not let the delete ever complete but he doesn't expect that is what you have based on the worked test.


No comments:

Post a Comment

linux - How to SSH to ec2 instance in VPC private subnet via NAT server

I have created a VPC in aws with a public subnet and a private subnet. The private subnet does not have direct access to external network. S...