From owner-freebsd-stable@FreeBSD.ORG Tue Sep 30 00:24:46 2014 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A0EA526A for ; Tue, 30 Sep 2014 00:24:46 +0000 (UTC) Received: from fs.denninger.net (wsip-70-169-168-7.pn.at.cox.net [70.169.168.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "NewFS.denninger.net", Issuer "NewFS.denninger.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 66203EAD for ; Tue, 30 Sep 2014 00:24:45 +0000 (UTC) Received: from [192.168.1.40] (localhost [127.0.0.1]) by fs.denninger.net (8.14.9/8.14.8) with ESMTP id s8U0OXQG099405 for ; Mon, 29 Sep 2014 19:24:33 -0500 (CDT) (envelope-from karl@denninger.net) Received: from [192.168.1.40] (TLS/SSL) [192.168.1.40] by Spamblock-sys (LOCAL/AUTH); Mon Sep 29 19:24:33 2014 Message-ID: <5429F820.6040305@denninger.net> Date: Mon, 29 Sep 2014 19:24:00 -0500 From: Karl Denninger User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.1.2 MIME-Version: 1.0 To: freebsd-stable@freebsd.org Subject: Re: MPS References: <5429BB41.8080609@denninger.net> <018BC41041EE4DF589EF5D834AD45BAD@multiplay.co.uk> In-Reply-To: Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha1; boundary="------------ms040804000906030202030207" X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Sep 2014 00:24:46 -0000 This is a cryptographically signed message in MIME format. --------------ms040804000906030202030207 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 9/29/2014 7:17 PM, Kevin Oberman wrote: > On Mon, Sep 29, 2014 at 1:25 PM, Steven Hartland > wrote: > >> I seem to remember detection being made parallel for speed, as there >> can be lots of drives. The Avago guys may be able to shed more light. >> >> Regards >> Steve >> >> ----- Original Message ----- From: "Karl Denninger" >> To: >> Sent: Monday, September 29, 2014 9:04 PM >> Subject: MPS >> >> >> >> Has anyone found a set of configuration settings for the mps driver (L= SI >> SAS series cards) that gets the following situation under control? >> >> I have a machine here with one of these cards and a SAS expander. The= >> expander has 16 ports, the card has 8 -- one connector attached to the= >> expander. So I have 20 potential drives associated with this adapter.= >> >> The issue is that the driver makes utterly no sense when it comes to >> assigning drive designations. In theory it should enumerate in series= ; >> that is, target 0 on the card should be da0, target 1 da1, etc. This >> assumes there are no gaps in the attached drives; if there are then >> things may move around, which is why some people wire drives. >> >> Ok so far, but it doesn't behave that way! >> >> This is the excerpt of what it returns: >> >> root@Dbms-10:/boot # dmesg|grep mps >> mps0: port 0xc000-0xc0ff mem >> 0xfbb3c000-0xfbb3ffff,0xfbb40000-0xfbb7ffff irq 30 at device 0.0 on pc= i3 >> mps0: Firmware: 19.00.00.00, Driver: 19.00.00.00-fbsd >> mps0: IOCCapabilities: >> 1285c >> ses0 at mps0 bus 0 scbus0 target 8 lun 0 >> da0 at mps0 bus 0 scbus0 target 0 lun 0 >> da1 at mps0 bus 0 scbus0 target 1 lun 0 >> da2 at mps0 bus 0 scbus0 target 2 lun 0 >> da3 at mps0 bus 0 scbus0 target 3 lun 0 >> *da4 at mps0 bus 0 scbus0 target 6 lun 0** >> **da5 at mps0 bus 0 scbus0 target 7 lun 0* >> >> Those last two are the drives the card identifies as being on targets = 0 >> and 1 when looked at in the BIOS. The other four drives (0-3) are on >> the SAS expander -- and there's a two drive "gap" in the targets >> identified as well which has zippo for logic associated with it. >> >> Even more-oddly if I swap the plugs -- that is, plug the two boot disk= s >> into the OTHER connector (putatively targets 4-7) and put the SAS >> expander on the 0-3 port connector the BIOS *does* show the target shi= ft >> but the machine, when it boots, still places those two drives on targe= ts >> 6 and 7! >> >> There is nothing in /boot/loader.conf or /boot/device.hints that relat= es >> to these devices at all. >> >> I would like to wire down the various ports so Drive Bay #x correspond= s >> to da[x] but how can you when you don't know what order they will come= >> up in predicated on either the physical port they're occupying or, for= >> that matter, what the card's BIOS shows them as? >> >> I looked in the driver's man page and saw nothing related to this; I g= et >> around it by labeling the drives and then using "glabel list" to see >> what da[x] number it grabbed if I need to pull a disk while the machin= e >> is up, but this is a rather-unfriendly way to handle that. >> >> -- >> Karl Denninger >> karl@denninger.net >> /The Market Ticker/ >> > Karl, > > This problem seems to crop up with some controllers, especially when p= ort > expanders are used. > > If at all possible, I really recommend labeling them and using the labe= ls > in lieu of device names. I stopped using device names some time ago jus= t > because of this. I do recommend using the labeling for the file system > involved as opposed to glabel, if it has one. Certainly UFS does and I > thought ZFS did, as well, but I am less sure of that as ZFS started to > stabilize about the time I stopped dealing with big systems. I still fi= nd > UFS quite adequate for my baby server and laptop. I do use glabel for s= wap. > > In any case, using labeling will give you stable names to put into thin= gs > like your fstab and not make you worry about the actual device number. = And, > of course, talking to the manufacturer is also a good idea, espacially = when > they don't say "FreeBSD? We don't support that.". > -- > R. Kevin Oberman, Network Engineer, Retired > E-mail: rkoberman@gmail.com > Yeah, that's been my answer along with lettering the drive carriers but it pisses me off. Ah well if that's how it is then that's how it is. :-)= Glabel works fine across the board provided you put it on a slice rather than a disk, which I do anyway so as to maintain "nice-nice" with AF (4k) disks along with leaving a small amount of space at the end unallocated, so if by some chance a different brand replacement drive isn't quite the same number of sectors it can be dropped into the pool as a replacement. --=20 Karl Denninger karl@denninger.net /The Market Ticker/ --------------ms040804000906030202030207 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIFTzCC BUswggQzoAMCAQICAQgwDQYJKoZIhvcNAQEFBQAwgZ0xCzAJBgNVBAYTAlVTMRAwDgYDVQQI EwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBM TEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExLzAtBgkqhkiG9w0BCQEWIGN1c3Rv bWVyLXNlcnZpY2VAY3VkYXN5c3RlbXMubmV0MB4XDTEzMDgyNDE5MDM0NFoXDTE4MDgyMzE5 MDM0NFowWzELMAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExFzAVBgNVBAMTDkthcmwg RGVubmluZ2VyMSEwHwYJKoZIhvcNAQkBFhJrYXJsQGRlbm5pbmdlci5uZXQwggIiMA0GCSqG SIb3DQEBAQUAA4ICDwAwggIKAoICAQC5n2KBrBmG22nVntVdvgKCB9UcnapNThrW1L+dq6th d9l4mj+qYMUpJ+8I0rTbY1dn21IXQBoBQmy8t1doKwmTdQ59F0FwZEPt/fGbRgBKVt3Quf6W 6n7kRk9MG6gdD7V9vPpFV41e+5MWYtqGWY3ScDP8SyYLjL/Xgr+5KFKkDfuubK8DeNqdLniV jHo/vqmIgO+6NgzPGPgmbutzFQXlxUqjiNAAKzF2+Tkddi+WKABrcc/EqnBb0X8GdqcIamO5 SyVmuM+7Zdns7D9pcV16zMMQ8LfNFQCDvbCuuQKMDg2F22x5ekYXpwjqTyfjcHBkWC8vFNoY 5aFMdyiN/Kkz0/kduP2ekYOgkRqcShfLEcG9SQ4LQZgqjMpTjSOGzBr3tOvVn5LkSJSHW2Z8 Q0dxSkvFG2/lsOWFbwQeeZSaBi5vRZCYCOf5tRd1+E93FyQfpt4vsrXshIAk7IK7f0qXvxP4 GDli5PKIEubD2Bn+gp3vB/DkfKySh5NBHVB+OPCoXRUWBkQxme65wBO02OZZt0k8Iq0i4Rci WV6z+lQHqDKtaVGgMsHn6PoeYhjf5Al5SP+U3imTjF2aCca1iDB5JOccX04MNljvifXgcbJN nkMgrzmm1ZgJ1PLur/ADWPlnz45quOhHg1TfUCLfI/DzgG7Z6u+oy4siQuFr9QT0MQIDAQAB o4HWMIHTMAkGA1UdEwQCMAAwEQYJYIZIAYb4QgEBBAQDAgWgMAsGA1UdDwQEAwIF4DAsBglg hkgBhvhCAQ0EHxYdT3BlblNTTCBHZW5lcmF0ZWQgQ2VydGlmaWNhdGUwHQYDVR0OBBYEFHw4 +LnuALyLA5Cgy7T5ZAX1WzKPMB8GA1UdIwQYMBaAFF3U3hpBZq40HB5VM7B44/gmXiI0MDgG CWCGSAGG+EIBAwQrFilodHRwczovL2N1ZGFzeXN0ZW1zLm5ldDoxMTQ0My9yZXZva2VkLmNy bDANBgkqhkiG9w0BAQUFAAOCAQEAZ0L4tQbBd0hd4wuw/YVqEBDDXJ54q2AoqQAmsOlnoxLO 31ehM/LvrTIP4yK2u1VmXtUumQ4Ao15JFM+xmwqtEGsh70RRrfVBAGd7KOZ3GB39FP2TgN/c L5fJKVxOqvEnW6cL9QtvUlcM3hXg8kDv60OB+LIcSE/P3/s+0tEpWPjxm3LHVE7JmPbZIcJ1 YMoZvHh0NSjY5D0HZlwtbDO7pDz9sZf1QEOgjH828fhtborkaHaUI46pmrMjiBnY6ujXMcWD pxtikki0zY22nrxfTs5xDWGxyrc/cmucjxClJF6+OYVUSaZhiiHfa9Pr+41okLgsRB0AmNwE f6ItY3TI8DGCBQowggUGAgEBMIGjMIGdMQswCQYDVQQGEwJVUzEQMA4GA1UECBMHRmxvcmlk YTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3RlbXMgTExDMRwwGgYD VQQDExNDdWRhIFN5c3RlbXMgTExDIENBMS8wLQYJKoZIhvcNAQkBFiBjdXN0b21lci1zZXJ2 aWNlQGN1ZGFzeXN0ZW1zLm5ldAIBCDAJBgUrDgMCGgUAoIICOzAYBgkqhkiG9w0BCQMxCwYJ KoZIhvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xNDA5MzAwMDI0MDBaMCMGCSqGSIb3DQEJBDEW BBR18phB1+c89/B+HhtpAiO++cKgYDBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFlAwQBKjAL BglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3DQMCAgFA MAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIG0BgkrBgEEAYI3EAQxgaYwgaMwgZ0xCzAJBgNV BAYTAlVTMRAwDgYDVQQIEwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoT EEN1ZGEgU3lzdGVtcyBMTEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExLzAtBgkq hkiG9w0BCQEWIGN1c3RvbWVyLXNlcnZpY2VAY3VkYXN5c3RlbXMubmV0AgEIMIG2BgsqhkiG 9w0BCRACCzGBpqCBozCBnTELMAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExEjAQBgNV BAcTCU5pY2V2aWxsZTEZMBcGA1UEChMQQ3VkYSBTeXN0ZW1zIExMQzEcMBoGA1UEAxMTQ3Vk YSBTeXN0ZW1zIExMQyBDQTEvMC0GCSqGSIb3DQEJARYgY3VzdG9tZXItc2VydmljZUBjdWRh c3lzdGVtcy5uZXQCAQgwDQYJKoZIhvcNAQEBBQAEggIAEaeYA8oFyq8UGPB1Fskf4eIrDA9U LP20qB8qq/05wYT36l63DIRHhhXKttOuJz67WPGEoHOyXdJV8kJqyQWh5EmoJByrPwizP8ln X24mFNEbklamnCSBwGtTRPdfpxn+uIev5mshiHBEv2rblq/nqZm76pt8s8c2aI0dz0qNmPBR +6cu+jte7pKRRV2hhs63zgCh5zlCep5F46SOtkfvb+6P1ZbVfZBhOadAlqHKvP+NHocW97bH aLB4E4ngzRdAYSLKJuD/KO5TSZS18QhAHN9ny7i9k8gngCXmx2BBftj5SKemoLEXATsWQzFK EqYwyUxnSCs91b/Bk0mInJI3nCQD5PpcxHXunpyVgjk5MWhx6ot04ezEFa5XYaA9yRLcn1Fp xl3AEFXV80/Wb7osffYOjpmJ0ehWvI26oODwEzSXdMDn+PI4ILAymQOcsS0YUWjGHnqBDELN AIes0yjG7IojbZK0f8zEBikpGL3jcTOLolFi8TRJNIn2jM2oXk7QNYhAXo5S/Ox4p98nsx+B x9u2u3SFmRfjyXd4Tx8oeJ5EIY+8quSwZm/WN6GmJmIw2UR+LHUp+VzbAerGSvfZz2UidguV 3XxCv3IwVFpB9gwBbNGVIvvAT5B1Ls2mGjbMY5AC/5KYUVvgozYfmoXzoxjQmcNXh3P22LcJ s7+l8X0AAAAAAAA= --------------ms040804000906030202030207--