ubuntu - ZFS L2ARC for Mirror Pool

I was reading up on L2ARC for my ZFS. I currently have two Pools, One for Data, and the other for Linux Containers and Virtual Machines. I would like some help with L2ARC, I know how to add it but was wonder if it would benefit me.

I'm running Ubuntu 16.04 with 6-8 LXC containers and sometimes a virtual machine or two in headless-virtualbox, I was wondering if adding a L2ARC will help with performance of my Virtual Machines and my LXC Containers.

Current Setup

pool: black
state: ONLINE

scan: scrub repaired 0 in 3h57m with 0 errors on Mon Jul 11 04:12:35 2016
config:

    NAME                                          STATE     READ WRITE CKSUM
    black                                         ONLINE       0     0     0
      mirror-0                                    ONLINE       0     0     0
        ata-WDC_WD10JFCX-68N6GN0_WD-WX91A65P0T3Z  ONLINE       0     0     0
        ata-WDC_WD10JFCX-68N6GN0_WD-WXA1EA4CSNJF  ONLINE       0     0     0

errors: No known data errors


pool: raid
state: ONLINE
scan: scrub repaired 0 in 26h32m with 0 errors on Tue Jul 12 04:04:11 2016
config:

    NAME                                          STATE     READ WRITE CKSUM
    raid                                          ONLINE       0     0     0
      raidz1-0                                    ONLINE       0     0     0
        ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N6LTTK5S  ONLINE       0     0     0

        ata-WDC_WD30EFRX-68EUZN0_WD-WCC4NJTT02XC  ONLINE       0     0     0
        ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N6LLTRUY  ONLINE       0     0     0
      raidz1-1                                    ONLINE       0     0     0
        ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N0UHPYUR  ONLINE       0     0     0
        ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N2ET6YU3  ONLINE       0     0     0
        ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N6TZJ7ZF  ONLINE       0     0     0

 errors: No known data errors

As you can see all my drives are WD Reds. I would add the L2ARC to Pool black, which is my LXC and Virtual Machine storage. I just did a write test to the pool and got 70-80MBPS of a 20GB file(video file), Read 80-90MBPS. I would add a 120GB or 240GB ssd to it as L2ARC.

Would this help the performance of the Pool for what I do?

Linux Containers; Emby Media Server, Apache, Web Dev Apache,Gitlab,Torrent,OpenWrt Dev,VPN Server,Mysql

Virtual Machines; Windows mainly for testing, odd Linux Server/Desktop for testing.

Specs
AMD Phenom™ II X6 1090T

32GB DDR3 Ram

Answer

Your pool black currently consists of two mirrored WD10JFCX drives, which according to Western Digital's spec sheet are:

• 5400 rpm (rotational latency about 5 ms on average, 11 ms worst case)


• 16 MB cache


• 1 TB


• 2.5" form factor


• SATA 6 Gb/s (SATA 3)

Neither the 5400 rpm rotation speed nor the puny 16 MB on-drive cache is conducive to highest performance, and the 1 TB is pretty small by today's standards as SATA drives go. 5400 rpm gives you, in theory, about 90 IOPS to the platters, and assuming that on average the platter will be half a revolution away from the data, its 11 ms rotation time gives you an average about 5 ms rotational latency.

Compare to this that you can get the Intel 540S series for about $308 apiece from Amazon (in the US), for which Intel gives the specs as:

• Up to 78,000 IOPS doing random reads, 85,000 IOPS doing random writes (latency 50 µs)


• Up to 560 MB/s sequential read, 480 MB/s sequential write


• 1 TB



• 2.5" form factor


• SATA 6 Gb/s (SATA 3)

Even if we play nice, you are looking at a 1,000-fold reduction in latency and a 1,000-fold increase in IOPS for about $600 and change. On a performance per dollar ratio, I'm pretty sure there's no beating that by adding L2ARC to your pool, and it's a drop in replacement. If you are cash-strapped, you can even replace one drive at a time with a corresponding SSD and still start reaping the benefits immediately.

Replacing those 5400 rpm disks with SSDs will do more to your black pool performance than any L2ARC you can realistically throw at it, almost regardless of your workload. What's more, while the Intel 540S series may or may not be a good fit for your requirements (see also here), even such a cheap SSD is almost certain to provide a better return on investment as compared to simply adding a L2ARC. If you want something higher spec'd, looking just at Intel, a DC S3500 1.2 TB SSD will set you back about $900 apiece, and a DC S3710 1.2 TB SSD will set you back about $1,000 apiece, both of which have similar performance but are likely to hold up better to use over time.

No endorsement of any specific products or resellers mentioned is either expressed or implied.