Date: Thu, 20 Jan 2011 10:00:46 -0700 From: "Kenneth D. Merry" <ken@freebsd.org> To: Borja Marcos <borjam@sarenet.es> Cc: freebsd-scsi@freebsd.org Subject: Re: mps questions Message-ID: <20110120170046.GA40879@nargothrond.kdm.org> In-Reply-To: <F3197986-1A54-461C-9D95-515348C6A827@sarenet.es> References: <F3197986-1A54-461C-9D95-515348C6A827@sarenet.es>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, Jan 20, 2011 at 17:47:21 +0100, Borja Marcos wrote: > > Hello, > > I''m trying some Dell PERC H200 cards on FreeBSD 8.2 with v28 ZFS. (Not a production environment, just running tests). > > I have two variants of them. One is an internal one, identified as this one. > > mps0: <LSI SAS2008> port 0xfc00-0xfcff mem 0xdf2b0000-0xdf2bffff,0xdf2c0000-0xdf2fffff irq 32 at device 0.0 on pci2 > mps0: Firmware: 02.15.63.00 > mps0: IOCCapabilities: 185c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,IR> > mps0: [ITHREAD] > > > And another one intended to conect to external SAS devices. > > mps1: <LSI SAS2008> port 0xec00-0xecff mem 0xdf4b0000-0xdf4bffff,0xdf4c0000-0xdf4fffff irq 41 at device 0.0 on pci5 > mps1: Firmware: 02.15.63.00 > mps1: IOCCapabilities: 185c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,IR> > mps1: [ITHREAD] > > Both have two SAS ports. And I am trying them on Dell PowerEdge R510 servers. I have 8 internal hard disks attached, and a Dell PowerVault MD1220 connected to one of the external SAS connectors. The MD1220 has 8 hard disks as well. > > I haven't configured any logical volumes on the cards, I just want unobtrusive SAS cards in order to create ZFS pools on raw disks, and the setup is working great, except for one problem: > > The *internal* card is showing just six hard disks under FreeBSD, not eight. > > da0 at mps0 bus 0 scbus0 target 1 lun 0 > da1 at mps0 bus 0 scbus0 target 2 lun 0 > da2 at mps0 bus 0 scbus0 target 3 lun 0 > da3 at mps0 bus 0 scbus0 target 4 lun 0 > da4 at mps0 bus 0 scbus0 target 5 lun 0 > da5 at mps0 bus 0 scbus0 target 6 lun 0 > > And this happens no matter how I connect the internal SAS connectors. There are two connectors from the disk backplanes to the card, SAS0 and SAS1 (I assume they are two buses) and even if I reverse them I still see eight disks. > I assume you mean you still see six disks... > However, when booting the system I see eight disks. The controller BIOS lists the backplane device and the eight disks, the FreeBSD boot loader says it sees eight disks, but FreeBSD sees just six. > > # camcontrol rescan all > Re-scan of bus 0 was successful > Re-scan of bus 1 was successful > # camcontrol devlist > <SEAGATE ST3300657SS-H EH02> at scbus0 target 1 lun 0 (da0,pass0) > <SEAGATE ST3300657SS-H EH02> at scbus0 target 2 lun 0 (da1,pass1) > <SEAGATE ST3300657SS-H EH02> at scbus0 target 3 lun 0 (da2,pass2) > <SEAGATE ST3300657SS-H EH02> at scbus0 target 4 lun 0 (da3,pass3) > <SEAGATE ST3300657SS-H EH02> at scbus0 target 5 lun 0 (da4,pass4) > <SEAGATE ST3300657SS-H EH02> at scbus0 target 6 lun 0 (da5,pass5) > <Alcor Flash Disk 8.07> at scbus1 target 0 lun 0 (da6,pass6) > > The card with the outside facing connectors shows the eight hard disks atttached to the MD1120 and the disks. A camcontrol devlist on the machine with both mps cards installed I see this: > > # camcontrol devlist > <SEAGATE ST3300657SS-H EH02> at scbus0 target 1 lun 0 (da0,pass0) > <SEAGATE ST3300657SS-H EH02> at scbus0 target 2 lun 0 (da1,pass1) > <SEAGATE ST3300657SS-H EH02> at scbus0 target 3 lun 0 (da2,pass2) > <SEAGATE ST3300657SS-H EH02> at scbus0 target 4 lun 0 (da3,pass3) > <SEAGATE ST3300657SS-H EH02> at scbus0 target 5 lun 0 (da4,pass4) > <SEAGATE ST3300657SS-H EH02> at scbus0 target 6 lun 0 (da5,pass5) > <SEAGATE ST9146852SS HT64> at scbus1 target 1 lun 0 (da6,pass6) > <SEAGATE ST9146852SS HT64> at scbus1 target 2 lun 0 (da7,pass7) > <SEAGATE ST9146852SS HT64> at scbus1 target 3 lun 0 (da8,pass8) > <SEAGATE ST9146852SS HT64> at scbus1 target 4 lun 0 (da9,pass9) > <SEAGATE ST9146852SS HT64> at scbus1 target 5 lun 0 (da10,pass10) > <SEAGATE ST9146852SS HT64> at scbus1 target 6 lun 0 (da11,pass11) > <SEAGATE ST9146852SS HT64> at scbus1 target 7 lun 0 (da12,pass12) > <SEAGATE ST9146852SS HT64> at scbus1 target 13 lun 0 (da13,pass13) > <DELL MD1220 1.01> at scbus1 target 36 lun 0 (ses0,pass14) > <UT165 USB Flash Disk 0.00> at scbus2 target 0 lun 0 (pass15,da14) > > > Any ideas? I was wondering if Dell had done something nasty to the internal card firmware (mps0 in my example), but seems that the eight internal disks are correctly presented to the BIOS. > If you see all the disks in the LSI BIOS, it most likely means there is a driver bug that is keeping FreeBSD from seeing all the disks. Try booting with -v (boot -v at the boot loader prompt) and send the full dmesg output. That will cause the driver to print some additional information when it probes. After that, try booting with hw.mps.N.debug_level=1 set in loader.conf, where N is either 0 or 1, depending on which mps instance is having the problem. You can turn it off via sysctl after you boot. Ken -- Kenneth Merry ken@FreeBSD.ORG
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20110120170046.GA40879>