Sunday, February 17, 2019

LSI RAID: Write cache policy affects read performance?

I have a server with a LSI MegaRAID SAS 9260-4i controller, RAID-5 with 3 x 2 TB disks. I did some performance testing (with iozone3) and the numbers show clearly that the write cache policy affects the read performance as well. If I set the policy to WriteBack I get about 2x the read performance in comparison with WriteThrough. How could the write cache affect the read performance?




Here are the details of the setup:



megacli -LDInfo -L0 -a0

Adapter 0 -- Virtual Drive Information:
Virtual Drive: 0 (Target Id: 0)
Name :
RAID Level : Primary-5, Secondary-0, RAID Level Qualifier-3
Size : 3.637 TB

Is VD emulated : Yes
Parity Size : 1.818 TB
State : Optimal
Strip Size : 512 KB
Number Of Drives : 3
Span Depth : 1
Default Cache Policy: WriteThrough, ReadAhead, Direct, No Write Cache if Bad BBU
Current Cache Policy: WriteThrough, ReadAhead, Direct, No Write Cache if Bad BBU
Default Access Policy: Read/Write
Current Access Policy: Read/Write

Disk Cache Policy : Disabled
Encryption Type : None
Bad Blocks Exist: No
Is VD Cached: No


With WriteBack enabled (everything else is unchanged):



Default Cache Policy: WriteBack, ReadAhead, Direct, Write Cache OK if Bad BBU
Current Cache Policy: WriteBack, ReadAhead, Direct, Write Cache OK if Bad BBU



Some numbers from iozone3:



 WriteThrough:
random random
KB reclen write rewrite read reread read write
2033120 64 91963 38146 144980 139122 11795 21564
2033120 128 83039 90746 118660 118147 21193 33686
2033120 256 78933 40359 113611 114327 31493 51838

2033120 512 71133 39453 131113 143323 28712 60946
2033120 1024 91233 76601 141257 142820 35869 45331
2033120 2048 58507 48419 136078 135220 51200 54548
2033120 4096 98426 70490 119342 134319 80883 57843
2033120 8192 70302 63047 132495 144537 101882 57984
2033120 16384 79594 29208 148972 135650 124207 79281

WriteBack:
random random
KB reclen write rewrite read reread read write

2033120 64 347208 302472 331824 302075 12923 31795
2033120 128 354489 343420 292668 322294 24018 45813
2033120 256 379546 343659 320315 302126 37747 71769
2033120 512 381603 352871 280553 322664 33192 116522
2033120 1024 374790 349123 289219 290284 43154 232669
2033120 2048 364758 342957 297345 320794 73880 264555
2033120 4096 368939 339926 303161 324334 128764 281280
2033120 8192 374004 346851 303138 326100 186427 324315
2033120 16384 379416 340577 284131 289762 254757 356530



Some details about the system:




  • Ubuntu 12.04

  • 64 bit

  • Kernel 3.2.0 (3.2.0-58-generic)

  • Memory was limited to 1 GB for the test

  • iozone3 version 397-2

  • Partition used for the test:

    /dev/sda4 /var ext4 rw,relatime,user_xattr,barrier=1,data=ordered 0 0

No comments:

Post a Comment

linux - How to SSH to ec2 instance in VPC private subnet via NAT server

I have created a VPC in aws with a public subnet and a private subnet. The private subnet does not have direct access to external network. S...