What filesystem should I use for a large amount of disk space in Linux (>32TB)?

I am trying to make a large file system in linux, but have run into problems with many of the common file systems.

• JFS has a bug that does not allow expansion over 32TB.


• XFS has a bug in the fsck that causes the machine to use all available memory and crash when running the fsck on a disk that has a large amount of data (~20TB).


• EXT4 is limited to 16TB due to a problem with e2fsprogs.


• BTRFS will be nice, but they do not currently have a fsck, which I will need.

Any other ideas?

Answer

It may not be as fast as the others, being only a userland fuse based system in Linux, but ZFS may fit the bill...

The name originally stood for "Zettabyte File System". The original name selectors happened to like the name, and a ZFS file system has the ability to store 2⁵⁸ zettabytes, where each ZB is 2⁷⁰ bytes.

ZFS is a 128-bit file system, so it can address 1.84 × 10¹⁹ times more data than 64-bit systems such as NTFS. The limitations of ZFS are designed to be so large that they would never be encountered. Some theoretical limits in ZFS are:

• 2⁴⁸ — Number of entries in any individual directory


• 16 exabytes (16×10¹⁸ bytes) — Maximum size of a single file


• 16 exabytes — Maximum size of any attribute



• 2⁵⁶ zettabytes (2⁷⁸ bytes) — Maximum size of any zpool


• 2⁵⁶ — Number of attributes of a file (actually constrained to 2⁴⁸ for the number of files in a ZFS file system)


• 2⁶⁴ — Number of devices in any zpool


• 2⁶⁴ — Number of zpools in a system


• 2⁶⁴ — Number of file systems in a zpool

There are some who say that there aren't enough atoms in the earth crust to make a file storage array big enough to exceed the limitations of ZFS.