Date: Mon, 29 Sep 2014 19:24:00 -0500 From: Karl Denninger <karl@denninger.net> To: freebsd-stable@freebsd.org Subject: Re: MPS Message-ID: <5429F820.6040305@denninger.net> In-Reply-To: <CAN6yY1v2MyOMahcTEagjxKQ33q56MmSwHx87_ZqyVYR5RDH8Mw@mail.gmail.com> References: <5429BB41.8080609@denninger.net> <018BC41041EE4DF589EF5D834AD45BAD@multiplay.co.uk> <CAN6yY1v2MyOMahcTEagjxKQ33q56MmSwHx87_ZqyVYR5RDH8Mw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
This is a cryptographically signed message in MIME format. --------------ms040804000906030202030207 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 9/29/2014 7:17 PM, Kevin Oberman wrote: > On Mon, Sep 29, 2014 at 1:25 PM, Steven Hartland <killing@multiplay.co.= uk> > wrote: > >> I seem to remember detection being made parallel for speed, as there >> can be lots of drives. The Avago guys may be able to shed more light. >> >> Regards >> Steve >> >> ----- Original Message ----- From: "Karl Denninger" <karl@denninger.ne= t> >> To: <freebsd-stable@freebsd.org> >> Sent: Monday, September 29, 2014 9:04 PM >> Subject: MPS >> >> >> >> Has anyone found a set of configuration settings for the mps driver (L= SI >> SAS series cards) that gets the following situation under control? >> >> I have a machine here with one of these cards and a SAS expander. The= >> expander has 16 ports, the card has 8 -- one connector attached to the= >> expander. So I have 20 potential drives associated with this adapter.= >> >> The issue is that the driver makes utterly no sense when it comes to >> assigning drive designations. In theory it should enumerate in series= ; >> that is, target 0 on the card should be da0, target 1 da1, etc. This >> assumes there are no gaps in the attached drives; if there are then >> things may move around, which is why some people wire drives. >> >> Ok so far, but it doesn't behave that way! >> >> This is the excerpt of what it returns: >> >> root@Dbms-10:/boot # dmesg|grep mps >> mps0: <LSI SAS2008> port 0xc000-0xc0ff mem >> 0xfbb3c000-0xfbb3ffff,0xfbb40000-0xfbb7ffff irq 30 at device 0.0 on pc= i3 >> mps0: Firmware: 19.00.00.00, Driver: 19.00.00.00-fbsd >> mps0: IOCCapabilities: >> 1285c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,HostD= isc> >> ses0 at mps0 bus 0 scbus0 target 8 lun 0 >> da0 at mps0 bus 0 scbus0 target 0 lun 0 >> da1 at mps0 bus 0 scbus0 target 1 lun 0 >> da2 at mps0 bus 0 scbus0 target 2 lun 0 >> da3 at mps0 bus 0 scbus0 target 3 lun 0 >> *da4 at mps0 bus 0 scbus0 target 6 lun 0** >> **da5 at mps0 bus 0 scbus0 target 7 lun 0* >> >> Those last two are the drives the card identifies as being on targets = 0 >> and 1 when looked at in the BIOS. The other four drives (0-3) are on >> the SAS expander -- and there's a two drive "gap" in the targets >> identified as well which has zippo for logic associated with it. >> >> Even more-oddly if I swap the plugs -- that is, plug the two boot disk= s >> into the OTHER connector (putatively targets 4-7) and put the SAS >> expander on the 0-3 port connector the BIOS *does* show the target shi= ft >> but the machine, when it boots, still places those two drives on targe= ts >> 6 and 7! >> >> There is nothing in /boot/loader.conf or /boot/device.hints that relat= es >> to these devices at all. >> >> I would like to wire down the various ports so Drive Bay #x correspond= s >> to da[x] but how can you when you don't know what order they will come= >> up in predicated on either the physical port they're occupying or, for= >> that matter, what the card's BIOS shows them as? >> >> I looked in the driver's man page and saw nothing related to this; I g= et >> around it by labeling the drives and then using "glabel list" to see >> what da[x] number it grabbed if I need to pull a disk while the machin= e >> is up, but this is a rather-unfriendly way to handle that. >> >> -- >> Karl Denninger >> karl@denninger.net <mailto:karl@denninger.net> >> /The Market Ticker/ >> > Karl, > > This problem seems to crop up with some controllers, especially when p= ort > expanders are used. > > If at all possible, I really recommend labeling them and using the labe= ls > in lieu of device names. I stopped using device names some time ago jus= t > because of this. I do recommend using the labeling for the file system > involved as opposed to glabel, if it has one. Certainly UFS does and I > thought ZFS did, as well, but I am less sure of that as ZFS started to > stabilize about the time I stopped dealing with big systems. I still fi= nd > UFS quite adequate for my baby server and laptop. I do use glabel for s= wap. > > In any case, using labeling will give you stable names to put into thin= gs > like your fstab and not make you worry about the actual device number. = And, > of course, talking to the manufacturer is also a good idea, espacially = when > they don't say "FreeBSD? We don't support that.". > -- > R. Kevin Oberman, Network Engineer, Retired > E-mail: rkoberman@gmail.com > Yeah, that's been my answer along with lettering the drive carriers but it pisses me off. Ah well if that's how it is then that's how it is. :-)= Glabel works fine across the board provided you put it on a slice rather than a disk, which I do anyway so as to maintain "nice-nice" with AF (4k) disks along with leaving a small amount of space at the end unallocated, so if by some chance a different brand replacement drive isn't quite the same number of sectors it can be dropped into the pool as a replacement. --=20 Karl Denninger karl@denninger.net <mailto:karl@denninger.net> /The Market Ticker/ --------------ms040804000906030202030207 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIFTzCC BUswggQzoAMCAQICAQgwDQYJKoZIhvcNAQEFBQAwgZ0xCzAJBgNVBAYTAlVTMRAwDgYDVQQI EwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBM TEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExLzAtBgkqhkiG9w0BCQEWIGN1c3Rv bWVyLXNlcnZpY2VAY3VkYXN5c3RlbXMubmV0MB4XDTEzMDgyNDE5MDM0NFoXDTE4MDgyMzE5 MDM0NFowWzELMAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExFzAVBgNVBAMTDkthcmwg RGVubmluZ2VyMSEwHwYJKoZIhvcNAQkBFhJrYXJsQGRlbm5pbmdlci5uZXQwggIiMA0GCSqG SIb3DQEBAQUAA4ICDwAwggIKAoICAQC5n2KBrBmG22nVntVdvgKCB9UcnapNThrW1L+dq6th d9l4mj+qYMUpJ+8I0rTbY1dn21IXQBoBQmy8t1doKwmTdQ59F0FwZEPt/fGbRgBKVt3Quf6W 6n7kRk9MG6gdD7V9vPpFV41e+5MWYtqGWY3ScDP8SyYLjL/Xgr+5KFKkDfuubK8DeNqdLniV jHo/vqmIgO+6NgzPGPgmbutzFQXlxUqjiNAAKzF2+Tkddi+WKABrcc/EqnBb0X8GdqcIamO5 SyVmuM+7Zdns7D9pcV16zMMQ8LfNFQCDvbCuuQKMDg2F22x5ekYXpwjqTyfjcHBkWC8vFNoY 5aFMdyiN/Kkz0/kduP2ekYOgkRqcShfLEcG9SQ4LQZgqjMpTjSOGzBr3tOvVn5LkSJSHW2Z8 Q0dxSkvFG2/lsOWFbwQeeZSaBi5vRZCYCOf5tRd1+E93FyQfpt4vsrXshIAk7IK7f0qXvxP4 GDli5PKIEubD2Bn+gp3vB/DkfKySh5NBHVB+OPCoXRUWBkQxme65wBO02OZZt0k8Iq0i4Rci WV6z+lQHqDKtaVGgMsHn6PoeYhjf5Al5SP+U3imTjF2aCca1iDB5JOccX04MNljvifXgcbJN nkMgrzmm1ZgJ1PLur/ADWPlnz45quOhHg1TfUCLfI/DzgG7Z6u+oy4siQuFr9QT0MQIDAQAB o4HWMIHTMAkGA1UdEwQCMAAwEQYJYIZIAYb4QgEBBAQDAgWgMAsGA1UdDwQEAwIF4DAsBglg hkgBhvhCAQ0EHxYdT3BlblNTTCBHZW5lcmF0ZWQgQ2VydGlmaWNhdGUwHQYDVR0OBBYEFHw4 +LnuALyLA5Cgy7T5ZAX1WzKPMB8GA1UdIwQYMBaAFF3U3hpBZq40HB5VM7B44/gmXiI0MDgG CWCGSAGG+EIBAwQrFilodHRwczovL2N1ZGFzeXN0ZW1zLm5ldDoxMTQ0My9yZXZva2VkLmNy bDANBgkqhkiG9w0BAQUFAAOCAQEAZ0L4tQbBd0hd4wuw/YVqEBDDXJ54q2AoqQAmsOlnoxLO 31ehM/LvrTIP4yK2u1VmXtUumQ4Ao15JFM+xmwqtEGsh70RRrfVBAGd7KOZ3GB39FP2TgN/c L5fJKVxOqvEnW6cL9QtvUlcM3hXg8kDv60OB+LIcSE/P3/s+0tEpWPjxm3LHVE7JmPbZIcJ1 YMoZvHh0NSjY5D0HZlwtbDO7pDz9sZf1QEOgjH828fhtborkaHaUI46pmrMjiBnY6ujXMcWD pxtikki0zY22nrxfTs5xDWGxyrc/cmucjxClJF6+OYVUSaZhiiHfa9Pr+41okLgsRB0AmNwE f6ItY3TI8DGCBQowggUGAgEBMIGjMIGdMQswCQYDVQQGEwJVUzEQMA4GA1UECBMHRmxvcmlk YTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3RlbXMgTExDMRwwGgYD VQQDExNDdWRhIFN5c3RlbXMgTExDIENBMS8wLQYJKoZIhvcNAQkBFiBjdXN0b21lci1zZXJ2 aWNlQGN1ZGFzeXN0ZW1zLm5ldAIBCDAJBgUrDgMCGgUAoIICOzAYBgkqhkiG9w0BCQMxCwYJ KoZIhvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xNDA5MzAwMDI0MDBaMCMGCSqGSIb3DQEJBDEW BBR18phB1+c89/B+HhtpAiO++cKgYDBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFlAwQBKjAL BglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3DQMCAgFA MAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIG0BgkrBgEEAYI3EAQxgaYwgaMwgZ0xCzAJBgNV BAYTAlVTMRAwDgYDVQQIEwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoT EEN1ZGEgU3lzdGVtcyBMTEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExLzAtBgkq hkiG9w0BCQEWIGN1c3RvbWVyLXNlcnZpY2VAY3VkYXN5c3RlbXMubmV0AgEIMIG2BgsqhkiG 9w0BCRACCzGBpqCBozCBnTELMAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExEjAQBgNV BAcTCU5pY2V2aWxsZTEZMBcGA1UEChMQQ3VkYSBTeXN0ZW1zIExMQzEcMBoGA1UEAxMTQ3Vk YSBTeXN0ZW1zIExMQyBDQTEvMC0GCSqGSIb3DQEJARYgY3VzdG9tZXItc2VydmljZUBjdWRh c3lzdGVtcy5uZXQCAQgwDQYJKoZIhvcNAQEBBQAEggIAEaeYA8oFyq8UGPB1Fskf4eIrDA9U LP20qB8qq/05wYT36l63DIRHhhXKttOuJz67WPGEoHOyXdJV8kJqyQWh5EmoJByrPwizP8ln X24mFNEbklamnCSBwGtTRPdfpxn+uIev5mshiHBEv2rblq/nqZm76pt8s8c2aI0dz0qNmPBR +6cu+jte7pKRRV2hhs63zgCh5zlCep5F46SOtkfvb+6P1ZbVfZBhOadAlqHKvP+NHocW97bH aLB4E4ngzRdAYSLKJuD/KO5TSZS18QhAHN9ny7i9k8gngCXmx2BBftj5SKemoLEXATsWQzFK EqYwyUxnSCs91b/Bk0mInJI3nCQD5PpcxHXunpyVgjk5MWhx6ot04ezEFa5XYaA9yRLcn1Fp xl3AEFXV80/Wb7osffYOjpmJ0ehWvI26oODwEzSXdMDn+PI4ILAymQOcsS0YUWjGHnqBDELN AIes0yjG7IojbZK0f8zEBikpGL3jcTOLolFi8TRJNIn2jM2oXk7QNYhAXo5S/Ox4p98nsx+B x9u2u3SFmRfjyXd4Tx8oeJ5EIY+8quSwZm/WN6GmJmIw2UR+LHUp+VzbAerGSvfZz2UidguV 3XxCv3IwVFpB9gwBbNGVIvvAT5B1Ls2mGjbMY5AC/5KYUVvgozYfmoXzoxjQmcNXh3P22LcJ s7+l8X0AAAAAAAA= --------------ms040804000906030202030207--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5429F820.6040305>