Saturday, January 17, 2015

filesystems - ZFS protection against bit rot and silent corruption without RAID



I have never encountered a disk failure in my whole life but I had a dozen of bit rot and silent corruptions. 'Normal' file systems and hardware RAID can do absolutely nothing about it. I have been using application-level parity redundancy to protect my most important data, but it's apparently not quite effective.



I know ZFS store checksum of the chunks in software RAID, which offers protection against bit rot and silent corruption. The problem is, I do not want RAID and that's off the table.




So is it possible to use filesystem-level parity without RAID or multi-disk systems ? For example by allocation part of the disk space to store the parity of the "chunks" instead of storing the parity of equal sized disks on another disk as in RAID5.



Thanks everyone for answering.The following is the conclusion based on the answers:



No, it's not possible to use ZFS parity without RAID, but it's possible to use mirror copy without RAID.


Answer



A checksum can only detect corruption. It cannot help you restore the data.



If you aren't willing to mirror your data on two (or more) vdevs, then you can set the copies ZFS property, which will tell ZFS to keep n copies of the data (1, 2 or 3). In case one copy is corrupted, the other can be used to recover the data.




$ zfs get copies srv
NAME PROPERTY VALUE SOURCE
srv copies 1 default

$ zfs set copies=2 srv


Keep in mind that this does not protect you from an actual disk failure (which can happen even if you have never experienced it before). This is not a backup. Make appropriate backups of your data.


No comments:

Post a Comment

linux - How to SSH to ec2 instance in VPC private subnet via NAT server

I have created a VPC in aws with a public subnet and a private subnet. The private subnet does not have direct access to external network. S...