From owner-freebsd-scsi@FreeBSD.ORG Fri Feb 17 09:07:11 2012 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 850B51065672 for ; Fri, 17 Feb 2012 09:07:11 +0000 (UTC) (envelope-from Kashyap.Desai@lsi.com) Received: from na3sys009aog118.obsmtp.com (na3sys009aog118.obsmtp.com [74.125.149.244]) by mx1.freebsd.org (Postfix) with ESMTP id A94438FC18 for ; Fri, 17 Feb 2012 09:07:10 +0000 (UTC) Received: from paledge01.lsi.com ([192.19.193.42]) (using TLSv1) by na3sys009aob118.postini.com ([74.125.148.12]) with SMTP ID DSNKTz4YvYhNLKpqe16/bFDCSmJPWVpJw+8J@postini.com; Fri, 17 Feb 2012 01:07:10 PST Received: from PALCAS01.lsi.com (128.94.213.117) by PALEDGE01.lsi.com (192.19.193.42) with Microsoft SMTP Server (TLS) id 8.3.213.0; Fri, 17 Feb 2012 04:12:08 -0500 Received: from inbexch02.lsi.com (135.36.98.40) by PALCAS01.lsi.com (128.94.213.117) with Microsoft SMTP Server (TLS) id 8.3.213.0; Fri, 17 Feb 2012 04:07:09 -0500 Received: from inbmail01.lsi.com ([135.36.98.64]) by inbexch02.lsi.com ([135.36.98.40]) with mapi; Fri, 17 Feb 2012 14:37:04 +0530 From: "Desai, Kashyap" To: Jason Wolfe Date: Fri, 17 Feb 2012 14:37:03 +0530 Thread-Topic: LSI2008 controller clobbers first disk with new LSI mps driver Thread-Index: AcztUP4qrknoHjsCRHen3pl0UJk/UAAARDCw Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: "freebsd-scsi@freebsd.org" Subject: RE: LSI2008 controller clobbers first disk with new LSI mps driver X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Feb 2012 09:07:11 -0000 Jason, Can you switch your mail client to "text mode". It is difficult to do inlin= e reply in html format. version 08.55.00.00, version 11.255.03.00-fbsd and ver= sion 13.00.00.00-fbsd detected SES device as Target Id =3D 8. This target Id is retrieved from FW. I am able to map my theory in your cas= e. Now, Only problem is when you use "Old mps FreeBSD developers" things are d= ifferent (and you would like to see same result using other driver versions= ). Here is a reason why behavior is different. Our older mps driver does not had mapping code available in few driver rele= ases. (see mps_mapping.c file) Because of that reason Driver will assign Target Id sequential (Internal se= quence) from Event received from FW. Summary is all driver behavior with FW 10.00.02.00 is as expected. Can you boot into verbose mode and send me log where it actual hangs ? Als= o what is your /etc/fstab entry. ` Kashyap From: Jason Wolfe [mailto:nitroboost@gmail.com] Sent: Friday, February 17, 2012 2:19 PM To: Desai, Kashyap Cc: freebsd-scsi@freebsd.org Subject: Re: LSI2008 controller clobbers first disk with new LSI mps driver Kashyap, Yes I've run all 4 drivers on the same machine with the 10.00.02.00 FW: Old mps from (I believe) FreeBSD developers: mps0: port 0xe000-0xe0ff mem 0xfbd3c000-0xfbd3ffff,0xfbd40000= -0xfbd7ffff irq 26 at device 0.0 on pci4 mps0: Firmware: 10.00.02.00 mps0: IOCCapabilities: 1285c mps0: [ITHREAD] da0 at mps0 bus 0 scbus0 target 1 lun 0 da1 at mps0 bus 0 scbus0 target 2 lun 0 da2 at mps0 bus 0 scbus0 target 3 lun 0 da3 at mps0 bus 0 scbus0 target 4 lun 0 da4 at mps0 bus 0 scbus0 target 5 lun 0 da5 at mps0 bus 0 scbus0 target 6 lun 0 da6 at mps0 bus 0 scbus0 target 7 lun 0 da8 at mps0 bus 0 scbus0 target 9 lun 0 da7 at mps0 bus 0 scbus0 target 8 lun 0 da10 at mps0 bus 0 scbus0 target 11 lun 0 da9 at mps0 bus 0 scbus0 target 10 lun 0 da11 at mps0 bus 0 scbus0 target 12 lun 0 ses0 at mps0 bus 0 scbus0 target 13 lun 0 ses0: Fixed Enclosure Services SCSI-5 device ses0: 600.000MB/s transfers ses0: Command Queueing enabled ses0: SCSI-3 SES Device mpslsi.ko binary from 2011: mpslsi0: port 0xe000-0xe0ff mem 0xfbd3c000-0xfbd3ffff,0xfbd40= 000-0xfbd7ffff irq 26 at device 0.0 on pci4 mpslsi0: Firmware: 10.00.02.00, Driver: 08.55.00.00 mpslsi0: IOCCapabilities: 1285c mpslsi0: [ITHREAD] ses0 at mpslsi0 bus 0 target 8 lun 0 ses0: Fixed Enclosure Services SCSI-5 device ses0: 600.000MB/s transfers ses0: Command Queueing Enabled ses0: SCSI-3 SES Device da0 at mpslsi0 bus 0 target 9 lun 0 da1 at mpslsi0 bus 0 target 10 lun 0 da2 at mpslsi0 bus 0 target 11 lun 0 da3 at mpslsi0 bus 0 target 12 lun 0 da4 at mpslsi0 bus 0 target 13 lun 0 da5 at mpslsi0 bus 0 target 14 lun 0 da6 at mpslsi0 bus 0 target 15 lun 0 da7 at mpslsi0 bus 0 target 16 lun 0 da8 at mpslsi0 bus 0 target 17 lun 0 da9 at mpslsi0 bus 0 target 18 lun 0 da10 at mpslsi0 bus 0 target 19 lun 0 11.255.03.00-fbsd: mps0: port 0xe000-0xe0ff mem 0xfbd3c000-0xfbd3ffff,0xfbd40000= -0xfbd7ffff irq 26 at device 0.0 on pci4 mps0: Firmware: 10.00.02.00, Driver: 11.255.03.00-fbsd mps0: IOCCapabilities: 1285c mps0: [ITHREAD] ses0 at mps0 bus 0 scbus0 target 8 lun 0 ses0: Fixed Enclosure Services SCSI-5 device ses0: 600.000MB/s transfers ses0: Command Queueing enabled ses0: SCSI-3 SES Device da0 at mps0 bus 0 scbus0 target 9 lun 0 da1 at mps0 bus 0 scbus0 target 10 lun 0 da2 at mps0 bus 0 scbus0 target 11 lun 0 da3 at mps0 bus 0 scbus0 target 12 lun 0 da4 at mps0 bus 0 scbus0 target 13 lun 0 da5 at mps0 bus 0 scbus0 target 14 lun 0 da6 at mps0 bus 0 scbus0 target 15 lun 0 da7 at mps0 bus 0 scbus0 target 16 lun 0 da8 at mps0 bus 0 scbus0 target 17 lun 0 da9 at mps0 bus 0 scbus0 target 18 lun 0 da10 at mps0 bus 0 scbus0 target 19 lun 0 Current 13.00.00.00-fbsd: mps0: port 0xe000-0xe0ff mem 0xfbd3c000-0xfbd3ffff,0xfbd40000= -0xfbd7ffff irq 26 at device 0.0 on pci4 mps0: Firmware: 10.00.02.00, Driver: 13.00.00.00-fbsd mps0: IOCCapabilities: 1285c mps0: [ITHREAD] da1 at mps0 bus 0 scbus0 target 10 lun 0 da0 at mps0 bus 0 scbus0 target 9 lun 0 da2 at mps0 bus 0 scbus0 target 11 lun 0 da4 at mps0 bus 0 scbus0 target 13 lun 0 da5 at mps0 bus 0 scbus0 target 14 lun 0 da3 at mps0 bus 0 scbus0 target 12 lun 0 da7 at mps0 bus 0 scbus0 target 16 lun 0 da8 at mps0 bus 0 scbus0 target 17 lun 0 da6 at mps0 bus 0 scbus0 target 15 lun 0 da9 at mps0 bus 0 scbus0 target 18 lun 0 da10 at mps0 bus 0 scbus0 target 19 lun 0 ses0 at mps0 bus 0 scbus0 target 8 lun 0 ses0: Fixed Enclosure Services SCSI-5 device ses0: 600.000MB/s transfers ses0: Command Queueing enabled ses0: SCSI-3 SES Device Jason On Fri, Feb 17, 2012 at 1:24 AM, Desai, Kashyap > wrote: Jason, Me also surprised when I see so many queries on . Really wonderful to know that there are good amount of driver use = as well. I was under impression that if you keep same FW and just change Driver, the= re should not be any difference in Target IDs assigned to Device connected = behind that HBA. Is this possible for you to keep everything unchanged and just change Drive= r version and see how things behaves. Please share "dmesg" logs of your bot= h drivers. BTW, did you tested "13.00.00.00-fbsd" and "11.00.00.00" on sa= me machine ? FYI: We never observe this kind of issue in our lab. ` Kashyap From: Jason Wolfe [mailto:nitroboost@gmail.com= ] Sent: Friday, February 17, 2012 1:31 PM To: Desai, Kashyap Cc: freebsd-scsi@freebsd.org Subject: Re: LSI2008 controller clobbers first disk with new LSI mps driver Kashyap, Ah a response from LSI, that's a pleasant surprise :) Everything you've st= ated looks correct to me, the FreeBSD developed driver that has been replac= ed by the LSI driver has no issues with either firmware. Your likely aware= , but just to confirm, here is the history of the 3 various LSI drivers tha= t have the issue on the 10.00.02.00 FW: 11.00.00.00 - binary driver I had received from you guys in mid 2011, mpsl= si.ko, one for each 7.2-RELEASE and 8.2-RELEASE 11.255.03.00-fbsd - initial LSI driver committed to 8-STABLE on 2/2, r23092= 2 13.00.00.00-fbsd - commited to 8-STABLE on 2/14, r231680 I have about 40 boxes with the 10.00.02.00 FW I've tested, so I'm fairly ce= rtain it's not bad hardware or a fluke. You guys haven't seen anything lik= e this in house? I'd hate to hear I have to update the FW on these boxes a= s they are all quite a ways from me, though it seems there is some way to w= ork around the behavior in the driver as the FreeBSD one does? I have a fe= w of these boxes out of service so I'm game to try some things out should t= hat help. Thank for the response, Jason On Thu, Feb 16, 2012 at 11:29 PM, Desai, Kashyap > wrote: Jason, I have gone through your data provided in this thread. It is well understoo= d because of your descriptive data. So What I understood here is: 1. You tested with HBA Fw "07.00.00.00" and "10.00.02.00" 2. you have run your test on two different LSI BIOS versions. Grabbed from below line. MPT2BIOS-7.11.00.00 (2010.07.29) / PRODUCT REVISION 7.00.00.00 MPT2BIOS-7.19.00.00 (2011.05.16) / PRODUCT REVISION 10.00.02.00 Now I am able to see below three difference in your setup. See FW version and check starting target id, all three has different way of= assigning TargetIDs. For first two case target id start with "8" but SES device assignment is di= fferent. Last case target id start with "1" mps0: Firmware: 07.00.00.00, Driver: 11.255.03.00-fbsd (OR) 13.00.00.00-fbs= d > > at scbus0 target 8 lun 0 (pass0,da0) > > at scbus0 target 9 lun 0 (pass1,da1) > > at scbus0 target 10 lun 0 (pass2,da2) > > at scbus0 target 11 lun 0 (pass3,da3) > > at scbus0 target 12 lun 0 (pass4,da4) > > at scbus0 target 13 lun 0 (pass5,da5) > > at scbus0 target 14 lun 0 (pass6,da6) > > at scbus0 target 15 lun 0 (pass7,da7) > > at scbus0 target 16 lun 0 (pass8,da8) > > at scbus0 target 17 lun 0 (pass9,da9) > > at scbus0 target 18 lun 0 (pass10,da10) > > at scbus0 target 19 lun 0 (pass11,da11) > > at scbus0 target 20 lun 0 (ses0,pass12) mps0: Firmware: 10.00.02.00, Driver: 13.00.00.00-fbsd mps0: Firmware: 10.00.02.00, Driver: 11.00.00.00 (OR) 8.2-STABLE Inbox > > at scbus0 target 8 lun 0 (ses0,pass0) > > at scbus0 target 9 lun 0 (da0,pass1) > > at scbus0 target 10 lun 0 (da1,pass2) > > at scbus0 target 11 lun 0 (da2,pass3) > > at scbus0 target 12 lun 0 (da3,pass4) > > at scbus0 target 13 lun 0 (da4,pass5) > > at scbus0 target 14 lun 0 (da5,pass6) > > at scbus0 target 15 lun 0 (da6,pass7) > > at scbus0 target 16 lun 0 (da7,pass8) > > at scbus0 target 17 lun 0 (da8,pass9) > > at scbus0 target 18 lun 0 (da9,pass10) > > at scbus0 target 19 lun 0 (da10,pass11) On the FBSD developed driver active in 8-STABLE prior to the LSI Release (Firmware: 10.00.02.00) > > at scbus0 target 1 lun 0 (pass0,da0) > > at scbus0 target 2 lun 0 (pass1,da1) > > at scbus0 target 3 lun 0 (pass2,da2) > > at scbus0 target 4 lun 0 (pass3,da3) > > at scbus0 target 5 lun 0 (pass4,da4) > > at scbus0 target 6 lun 0 (pass5,da5) > > at scbus0 target 7 lun 0 (pass6,da6) > > at scbus0 target 8 lun 0 (pass7,da7) > > at scbus0 target 9 lun 0 (pass8,da8) > > at scbus0 target 10 lun 0 (pass9,da9) > > at scbus0 target 11 lun 0 (pass10,da10) > > at scbus0 target 12 lun 0 (pass11,da11) > > at scbus0 target 13 lun 0 (ses0,pass12) In summary, (please confirm) 1. you have not seen any issue if you use "07.00.00.00" FW version. 2. _but_ when you use "10.00.02.00" FW, with "13.00.00.00-fbsd" driver vers= ion you are seeing SES is detected before Drives as pass0. 3. When you use "10.00.02.00" FW with 8-STABLE inbox FBSD driver, you are f= inding SES device detected after Drives. All driver is doing here is asking CAM layer to scan Bus when there is any = device added on that bus. So depending upon actual target Id assigned by FW, it will be detected to = camcontrol. So bottom line is FW plays major role in sequencing Drives behind LSI contr= oller. ~ Kashyap