I just added a LSI 9211-8i to a system running Debian Wheezy (on the Linux kernel). All software is up to date and the kernel is 3.2.65-1+deb7u2 x86_64 according to uname.
The card came straight out of the packaging and into the host after visual inspection that didn't uncover anything that was clearly wrong with the card (though I have no known good card to compare against). This, along with the fact that the kernel is speaking to the card (see below) leads me to believe that the card itself is slightly more useful than a dud.
Physically installing the card posed no problems. The card being PCIe x8 didn't need the full length of the PCIe x16 slot I had available, but as far as I can tell that should not be a problem if the host and card are speaking to each other at all. The motherboard has two PCIe x16 slots, one of which is listed as "x4 performance". Since the card is obviously being detected at some level, I do not believe anything like the graphics-card-only x16 slots is at play here.
To the 9211's internal ports I hooked up two 8077-to-4x8482 breakout cables, connecting each to two HDDs (leaving unused the other two plugs on each) with no PMP or anything similar in between. One of the two 8077 ports (in the unlikely case it makes a difference, the one farther from the PCIe slot) was slightly finicky, but the cable clicked into and locked in place without arguments once I slided it in at the right angle. I looked more closely around the area of that port but could find no evidence of physical damage to the card.
The system was noticably noisier on boot compared to what it was before I installed these new drives, which leads me to believe that the card is, at the very least, supplying power and spinning up the drives. The drives subsequently spun down.
I expected the card to make some utterances during the boot process, and was rather surprised to get nothing of the sort (no "Press Ctrl-C to start LSI Logic Configuration Utility" prompt). I looked through the motherboard's BIOS setup, but could find no relevant switches that needed to be flipped for off-board BIOSes or HBAs. Hammering Ctrl+C during the boot process up to GRUB (to try to invoke the card's on-board configuration utility) did not produce any visible results.
The mpt2sas module was loaded automatically on boot, and seems to talk to the card just fine:
[ 1.692606] mpt2sas version 10.100.00.00 loaded
[ 1.698699] mpt2sas 0000:08:00.0: enabling device (0000 -> 0002)
[ 1.698717] mpt2sas 0000:08:00.0: setting latency timer to 64
[ 1.698721] mpt2sas0: 64 BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem (32967612 kB)
[ 1.698761] mpt2sas0: IO-APIC enabled: IRQ 16
[ 1.698764] mpt2sas0: iomem(0x00000000d0440000), mapped(0xffffc90013ea8000), size(16384)
[ 1.698766] mpt2sas0: ioport(0x0000000000001000), size(256)
[ 2.139165] mpt2sas0: Allocated physical memory: size(3379 kB)
[ 2.139168] mpt2sas0: Current Controller Queue Depth(1483), Max Controller Queue Depth(1720)
[ 2.139170] mpt2sas0: Scatter Gather Elements per IO(128)
[ 2.360461] mpt2sas0: LSISAS2008: FWVersion(20.00.00.00), ChipRevision(0x03), BiosVersion(07.27.01.00)
[ 2.360464] mpt2sas0: Protocol=(Initiator), Capabilities=(Raid,TLR,EEDP,Snapshot Buffer,Diag Trace Buffer,Task Set Full,NCQ)
[ 2.360563] mpt2sas0: sending port enable !!
[ 4.895613] mpt2sas0: host_add: handle(0x0001), sas_addr(0x500605b00963d470), phys(8)
[ 10.024028] mpt2sas0: port enable: SUCCESS
lspci shows that the card is being detected and identified:
$ lspci | grep LSI
08:00.0 Serial Attached SCSI controller: LSI Logic / Symbios Logic SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon] (rev 03)
$
However, and this is where it gets interesting, neither lsblk nor udevadm info --exportdb shows any of the new HDDs, insofar as I can tell. They are also (obviously, given udevadm) not showing up in any of the /dev/disk/by-* directories.
I tried running udevadm trigger just in case there was something iffy with the boot sequence ordering, but that did not change anything and did not add anything at all to the system log (i.e., the most recent portion of the output of dmesg was the same before and after running that command).
I am not inclined to believe that both of the brand new breakout cables are somehow broken.
Physically unplugging both of the breakout cables from the card (to remove the HDDs and cables from consideration in the case) did not make any discernable difference.
I followed these instructions to install the most recent version of MegaRAID Storage Manager on my system. (Basically, take the rpms, use alien --scripts to convert them to debs, and then dpkg --install the debs.) After that, with the drives plugged in and /etc/init.d/vivaldiframeworkd started, running /usr/local/MegaRAID Storage Manager/StorCLI/storcli64 show all prints the following:
Status Code = 0
Status = Success
Description = None
Number of Controllers = 0
Host Name = my-host
Operating System = Linux3.2.0-4-amd64
At this point I am somewhat running out of ideas. If there's any other information I can provide that might help answering this, just let me know. I'm almost starting to think that this is somehow a motherboard issue after all.
With the ultimate goal of using them for a ZFS pool, what incantations, magic utterances, sacrifices or other relevant rituals do I need to perform for the drives connected to the 9211 to show up in Linux?
UPDATE: After physically switching places of the graphics card and the 9211, the 9211's BIOS now shows up on boot and I was able to enter the configuration utility. It still shows no disks attached (even in the SAS Topology view), however, despite disks very definitely being attached and cables firmly seated on both ends. (I have not, however, created any RAID array using the card's configuration utility.) What's more is that the card reports that it has been "disabled". At this point I'm almost willing to chalk down my initial problems to a crappy motherboard, and my current problems to IR vs IT firmware on the 9211 itself. I will try flashing the card to IT firmware later and see how that goes; I plan on using IT firmware anyway because of ZFS, so there's no harm to doing so that I can see.
Answer
There turned out to be two actual problems (and one minor annoyance) involved in this. Many thanks to ewwhite for providing me with troubleshooting suggestions that eventually allowed me to fix this.
The first problem was that the PCI Express slot I installed the LSI in for whatever reason did not work fully with the HBA. What confused me here was mainly that Linux was detecting the card, but not doing much more (and certainly not detecting any of the disks attached to it). When I switched places between the graphics card and the HBA, the HBA's BIOS came up and I was able to enter the configuration utility as per ewwhite's instructions, and graphics is still working fine. (I guess that's what I get for working with what I have...) However, still no drives were showing up either in the configuration utility's "SAS Topology" menu, or in Linux, despite my triple-checking that the connections were secure.
At one point the controller was reporting itself as being "disabled by user"; this was related to the boot support setting in its configuration. Setting it to one of the "enabled" choices fixed that. This was the minor annoyance.
The second problem was mainly a misunderstanding on my part. I somehow thought that the card's 8077 ports would provide power for the drives, but they didn't. In retrospect, I suppose that makes sense. Connecting the power cable attached to the respective 8482 connector for the drives I was using to the system's power supply fixed that easily enough. (However, I have no good explanation for the initial noise when I powered the system up for the first time after installing the LSI.)
Having sorted out the above, the controller now works perfectly and the disks are currently being provisioned. I'm leaving this here in the hope that someone, some time, will find it useful and not make the same mistakes that I did.
No comments:
Post a Comment