NAS Performance: NFS vs Samba vs GlusterFS

I am planning my new shared storage infrastructure for a small webserver farm. Therefore i ran a lot of tests with a lot of NAS filesystems. By doing so, i got some unexpected results and would like to know whether somebody here can confirm this.

In short: Samba is hugely faster than NFS and GlusterFS for small file writes.

Here what i did:
I ran a simple "rsync benchmark" with a lot of files to compare the write performance for small files. For easier reproduction, i re-ran it just now with the contents of the current wordpress tar.gz.

• GlusterFS replicated 2: 32-35 seconds, high CPU load


• GlusterFS single: 14-16 seconds, high CPU load


• GlusterFS + NFS client: 16-19 seconds, high CPU load



• NFS kernel server + NFS client (sync): 32-36 seconds, very low CPU load


• NFS kernel server + NFS client (async): 3-4 seconds, very low CPU load


• Samba: 4-7 seconds, medium CPU load


• Direct disk: < 1 second

I am absolutely no samba guru (i think my last contact was with samba 2.x), so i did not optimize anything here - just out-of-the-box config (debian/squeeze package). The only thing i added "sync always = yes" which is supposed to enforce sync after write (but seeing those results..). Without it, the tests where about 1-2 seconds faster.

All the tests where run on the same machine (self-mounted it's NAS export), so no network delays - pure protocol performance.

Side node: As file system i used ext4 and xfs. The above results are with ext4. xfs performed up to 40% (less time) better. The machines are EC2 m1.small instances. NAS export are on EBS volumes, sources (extracted tar) on ephemeral disk.

So here we go: Can anybody explain to me why samba is so much faster?

Also: Is NFS performance with the kernel server supposed to be that horrible (outperformed by the GlusterFS NFS server) in sync mode? Any idea how to tune this?

Thanks, L