From owner-freebsd-scsi@FreeBSD.ORG Tue Oct 7 22:43:36 2014 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D9590328 for ; Tue, 7 Oct 2014 22:43:36 +0000 (UTC) Received: from exprod7og115.obsmtp.com (exprod7og115.obsmtp.com [64.18.2.217]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 84964BE6 for ; Tue, 7 Oct 2014 22:43:36 +0000 (UTC) Received: from mail-pa0-f44.google.com ([209.85.220.44]) (using TLSv1) by exprod7ob115.postini.com ([64.18.6.12]) with SMTP ID DSNKVDRskdC7ZQ7HGlg/TQBWPDwGaOM78XdI@postini.com; Tue, 07 Oct 2014 15:43:36 PDT Received: by mail-pa0-f44.google.com with SMTP id et14so7927972pad.31 for ; Tue, 07 Oct 2014 15:43:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:references:in-reply-to:mime-version :thread-index:date:message-id:subject:to:cc:content-type; bh=cuiV9/2ZvCk3bjaQflQLoipjJ5akzoOFmY6JptGjTow=; b=be25BZqw4DQeIN/A7Xl2R1WgRznTNoG137q6ZdfMP47TcdasDjNT6kcJcfiJIA+rSn bdezCeQXQSgW/8oXrfpoZ15YDtUed1XNKrVEfK8vi3Wc7BDM9foirAu0PESWvGTvjOh/ P93Ls7NzwAHxhyX1lkeBJZb/yCseXI8JM4GZKr0WkheukMgLKqev/uiYSVKFQ5lAd4Ri uSb2/Im7ExYNGABqQT2yLL+2QYHBNVKFb9KPi5+inHkSr8X3q9AaSjhUHLMkBdADGAcU akeH6VlJr207KQppTgloyl3ujUuTMJiveO1y0X0N4YjNL3u2UmqvHBhRFLEPSOLCPMTN Y1kQ== X-Gm-Message-State: ALoCoQnNNXOYIG9GvNXs2aCz9dkG+y9ALl34abOs5S1MaVkehHauGzfH32qGc26Lzm/etc4FrCJPCGKVQe3/Ov2f5YtQq6aFSndu3FOoCv022zWzez5SuV0GYTlNspA3Ymoye2OgCMfVMafJ8ZUkRpInmxtgVhlYtA== X-Received: by 10.66.220.194 with SMTP id py2mr6295193pac.81.1412721809423; Tue, 07 Oct 2014 15:43:29 -0700 (PDT) X-Received: by 10.66.220.194 with SMTP id py2mr6295181pac.81.1412721809251; Tue, 07 Oct 2014 15:43:29 -0700 (PDT) From: Stephen Mcconnell References: <54343A0F.6080905@interlog.com> In-Reply-To: <54343A0F.6080905@interlog.com> MIME-Version: 1.0 X-Mailer: Microsoft Outlook 14.0 Thread-Index: AQHAvPDqV/NT6Mjmvn39KZFmH6q6QwHpuJfJAoXUFXgBjNvBvwHFdtxGAVPo6gCb+tZ6YA== Date: Tue, 7 Oct 2014 16:43:30 -0600 Message-ID: Subject: RE: mpr vs mps performance To: dgilbert@interlog.com, Borja Marcos , Alan Somers Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD-scsi X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Oct 2014 22:43:36 -0000 > -----Original Message----- > From: Douglas Gilbert [mailto:dgilbert@interlog.com] > Sent: Tuesday, October 07, 2014 1:08 PM > To: Borja Marcos; Alan Somers > Cc: Stephen Mcconnell; FreeBSD-scsi > Subject: Re: mpr vs mps performance > > On 14-10-07 12:19 PM, Borja Marcos wrote: > > > > On Oct 7, 2014, at 6:11 PM, Alan Somers wrote: > > > >>> Borja, can you upgrade your card to the latest FW. Your version is > >>> really old (1.00.03.00). We should be up to 7.00.00.00 now. There > >>> could be a lot of fixes that would affect performance. > >> > >> To update the firmware, you must download LSI's > >> Installer_P6_for_FreeBSD from the link below. Then the tricky part: > >> you must hexedit the executable and replace all "mps3_" strings with > >> "mpr", suitably padded with NULLs. It sucks, I know; complain to > >> LSI/Avago. Then you must download the > >> 9300_8e_Package_P6_IT_Firmware_BIOS_for_MSDOS_Windows (or > whichever > >> is appropriate for your hardware) to get the actual firmware files. > >> Then you can use the hacked FreeBSD installer binary to download the > >> firmware from the Windows firmware package. > >> > >> http://www.lsi.com/support/pages/download-results.aspx?component=Stor > >> > age+Component&productfamily=Host+Bus+Adapters&productcode=P00766&as > se > >> ttype=0&productname=LSI+SAS+9300-8e > > > > Thanks, I'll try. Probably the SAS2 cards will go into production but I'm happy > to help with some tests. > > > > The performance problem was not sustained, but writing performance stalled > for several seconds now and then. It did not happen with the SAS2 HBA (which is > a Dell branded as H200 flashed to IT firmware). > > I'm not seeing a big difference between FreeBSD and Linux using a LSI SAS-3 HBA > with a 12 Gbps SAS SSD. > > With the same 9300-4i HBA (P6 firmware (where is P7 that Stephen mentioned > ??)) and the same disk (SEAGATE ST200FM0073 0005), using Linux as the client: > > Via iSCSI against FreeNAS 9.3-M4 (nightly build, 3 days old): > # ddpt --odx if=/dev/sg5 bs=512 of=/dev/sg5 seek=4m count=4m > 4194304+0 records in > 4194304+0 records out > time to transfer data: 11.275797 secs at 190.45 MB/sec > > On Linux lk 3.17.0 against the same disk, directly connected: > # ddpt if=/dev/sg2 bs=512 of=/dev/sg2 seek=4m count=4m > 4194304+0 records in > 4194304+0 records out > time to transfer data: 10.807345 secs at 198.71 MB/sec > > > The SSD is probably not optimized for an "up-copy" like that. With raw READs it > yields better than 700 MB/sec, using both ports it yields better than 900 MB/sec. > > "--odx" is a Microsoft term for a token based remote copy. It was proposed to > T10 as "XCOPYv2: Extended Copy Plus & Lite" and the result is found in SBC-3. > > Doug Gilbert Yesterday I mentioned that most of the performance gains on the Gen3 card come from MSI-x support. In fact, MSI-x Firmware support does add to performance, but I forgot about the actual hardware I/O acceleration in the Gen3 cards (Fast Path I/O), which increases performance significantly. Just wanted to add that. Also, I did a little testing and I got about the same numbers as Doug with the 3008 and a 12Gb SAS SSD (a little better than 700 MB/sec). With the 2008, I saw raw reads of around 425 MB/sec. I don't do a lot of benchmark testing, but that's what I saw in my little test. Steve > > >