Tuesday, October 30, 2018

zfs - What is the best private cloud storage setup

I need to create a private cloud and I'm searching for the best setup.
These are my 2 most important requirements
1. Disk and system redundant
2. Price / GB as low as possible



The system is going to be used as backup setup which will receive data 24/7 over SFTP and rsync. High throughput is not that important.



I'm planning to use glusterfs and consumer grade 4TB hard-drives.



I have worked out 3 possible setups





  1. 3 servers with 11 4TB HDD
    Setup up a replica 3 glusterfs and setup each hard drive as a separate ext4 brick.
    Total capacity: 44TB
    HDD / TB ratio of 0.75 (33HDD / 44TB)


  2. 2 servers with 11 4TB HDD
    The 11 hard-drives are combined in a RAIDZ3 ZFS storage pool. With a replica 2 gluster setup.
    Total capacity: 32TB (+ zfs compression)
    HDD / TB ratio of 0.68 (22HDD / 32TB)



  3. 3 servers with 11 4TB consumer hard-drives
    Setup up a replica 3 glusterfs and setup each hard-drive as a separate zfs storage pool and export each pool as a brick.
    Total capacity: 32TB (+ zfs compression)
    HDD / TB ratio of 0.68 (22HDD / 32TB) (Cheapest)




My remarks and concerns:
If a hard drive fails which setup will recover the quickest? In my opinion setup 1 and 3 because there only the contents of 1 hard-drive needs to be copied over the network. Instead of setup 2 were the hard-drive needs te be reconstructed by reading the parity of all the other harddrives in the system.
Will a zfs pool on 1 harddrive give me extra protection against for example bit rot?
With setup 1 and 3 I can loose 2 systems and still be up and running with setup 2 I can only loose 1 system.

When I use ZFS I can enable compression which will give me some extra storage.

No comments:

Post a Comment

linux - How to SSH to ec2 instance in VPC private subnet via NAT server

I have created a VPC in aws with a public subnet and a private subnet. The private subnet does not have direct access to external network. S...