sata - Best performance: 1tb vs larger drives in app using only sequential read/writes

I have a custom app that is multi-threaded; each thread runs on its own logical core (the workstation is a dual xeon, with 12 physical and 24 logical cores). So there are 24 threads running simultaneously.

I've been researching the multitude of storage options over the last 2 days, and my head is spinning at about 15k rpm.

The app has 2 modes, and they are exclusive: read data or write data; by this I mean that they won't be doing reads/writes interleaved. Each thread will just be doing long sequential reads or writes. The total storage I will need is huge: over 50 tbs (if you are reading this in the year 2016 you are probably having a good chuckle right now over the word "huge".)

Each thread will be reading or writing a file that is about .8tb

I am going to go with jbod, because if a drive fails, all I need to do is swap it, and the app will re-create the data in about 10 minutes.

I will be putting the drives into an external tower or rack, using a SATA OR SAS controller (haven't figured out the +/- of those yet).

So, my question: am I correct in assuming that using 1tb drives for this particular app would be better performance wise than using drives 2, 3 or 4 times that size? It would seem that, unless a 3 tb drive has sequential read/write throughput that is 3x that of a 1 tb drive, the smaller drive is the way to go.

Obviously, using 3 tb drives reduces the number of drives I need to worry about by 1/3, but that would only be a consideration if I could achieve performance that is in the same ballpark as the 1tb drives.

Answer

Most importantly, for best performance you need 24 drives (or more, bear with me), because you have 24 threads. If there are less disks than threads, you do not have sequential operation. Considering two threads on a single disk, it will have some seeks; each seek is say 10 ms, so a loss of about 1 MB of transfer opportunity. With only 10 seeks per second, you have 90 MB/s (2 x 45) instead of 1 x 100 MB/s.

I think you would be better with 48 drives 1 TB each. Drives would be paired, to get 24 stripped (aka RAID0) groups. You can assume that stripping done on OS level will have an unnoticeable impact, so effectively each thread gets double throughput of a 1 TB drive.

I see no possible benefit of 3 TB drives performance-wise.

Still the biggest performance benefit would be something totally different. Just make sure that the application streams the data effectively - that it seriously feeds the queue on the HDD with those I/O commands. If it is written in a way that it clumsily waits for some I/O to get completed before queueing another I/O, then it would severe the throughput.

• Best to fix it in the application itself or...



• To partly alleviate this, you would need to invest a lot of money into a disk array with a huge extra cache.

PS. Love the remark about the 2016, hi there folks! Did you get those hover boards already?