Thursday, June 21, 2018

Poor software raid10 read performance on Linux

I've recently set up new Linux based file server. The distribution I'm using is Ubuntu 10.10. I've created two software raid devices as follows:



mc@vmr:~$ cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md0 : active raid1 sdf1[1] sdh1[3] sde1[0] sdg1[2]
19530688 blocks [4/4] [UUUU]

md1 : active raid10 sdh3[3] sdf3[1] sde3[0] sdg3[2]
1912461184 blocks 64K chunks 2 near-copies [4/4] [UUUU]



Device /dev/md0 (raid1) is mounted on "/" and /dev/md1 (raid10) is mounted on "/home".
Unfortunately the performance of my raid10 is deeply unsatisfying.
Here is the performance of each single HDD:



mc@vmr:~$ sudo hdparm -t /dev/sdh3  
/dev/sdh3:
Timing buffered disk reads: 410 MB in 3.00 seconds = 136.57 MB/sec


mc@vmr:~$ sudo hdparm -t /dev/sdf3
/dev/sdf3:
Timing buffered disk reads: 402 MB in 3.01 seconds = 133.60 MB/sec

mc@vmr:~$ sudo hdparm -t /dev/sde3
/dev/sde3:
Timing buffered disk reads: 418 MB in 3.01 seconds = 139.10 MB/sec

mc@vmr:~$ sudo hdparm -t /dev/sdg3
/dev/sdg3:

Timing buffered disk reads: 406 MB in 3.00 seconds = 135.32 MB/sec


So naturally I was expecting read performance around 260 MB/sec, but instead I've got this:



mc@vmr:~$ sudo hdparm -t /dev/md1
/dev/md1:
Timing buffered disk reads: 172 MB in 3.04 seconds = 56.64 MB/sec



Firstly I've assumed that hdparm testing method is not to be 100% trusted so I did kind of real-world read test and performance was still not as expected (random.bin is placed on raid10):



mc@vmr:~$ dd if=random.bin of=/dev/null bs=256k
38800+0 records in
38800+0 records out
10171187200 bytes (10 GB) copied, 96.3874 s, 106 MB/s


Lastly I would like to share that the read performance of raid1 is exactly as expected:




mc@vmr:~$ sudo hdparm -t /dev/md0
/dev/md0:
Timing buffered disk reads: 412 MB in 3.01 seconds = 136.91 MB/sec


Has anyone came across problem like this? Any clues?

No comments:

Post a Comment

linux - How to SSH to ec2 instance in VPC private subnet via NAT server

I have created a VPC in aws with a public subnet and a private subnet. The private subnet does not have direct access to external network. S...