From owner-freebsd-scsi@FreeBSD.ORG Mon May 14 11:08:49 2007 Return-Path: X-Original-To: freebsd-scsi@FreeBSD.org Delivered-To: freebsd-scsi@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 51B8316A400 for ; Mon, 14 May 2007 11:08:49 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [69.147.83.40]) by mx1.freebsd.org (Postfix) with ESMTP id 4204713C458 for ; Mon, 14 May 2007 11:08:49 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (linimon@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.4/8.13.4) with ESMTP id l4EB8nkY033135 for ; Mon, 14 May 2007 11:08:49 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.13.4/8.13.4/Submit) id l4EB8lXd033131 for freebsd-scsi@FreeBSD.org; Mon, 14 May 2007 11:08:47 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 14 May 2007 11:08:47 GMT Message-Id: <200705141108.l4EB8lXd033131@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: linimon set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-scsi@FreeBSD.org Cc: Subject: Current problem reports assigned to you X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 May 2007 11:08:49 -0000 Current FreeBSD problem reports Critical problems Serious problems S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/39388 scsi ncr/sym drivers fail with 53c810 and more than 256MB m o kern/40895 scsi wierd kernel / device driver bug o kern/52638 scsi [panic] SCSI U320 on SMP server won't run faster than s kern/57398 scsi [mly] Current fails to install on mly(4) based RAID di o kern/60598 scsi wire down of scsi devices conflicts with config o kern/60641 scsi [sym] Sporadic SCSI bus resets with 53C810 under load s kern/61165 scsi [panic] kernel page fault after calling cam_send_ccb o kern/74627 scsi [ahc] [hang] Adaptec 2940U2W Can't boot 5.3 o kern/81887 scsi [aac] Adaptec SCSI 2130S aac0: GetDeviceProbeInfo comm o kern/90282 scsi [sym] SCSI bus resets cause loss of ch device o kern/92798 scsi [ahc] SCSI problem with timeouts o kern/93128 scsi [sym] FreeBSD 6.1 BETA 1 has problems with Symbios/LSI o kern/94838 scsi Kernel panic while mounting SD card with lock switch o o kern/99954 scsi [ahc] reading from DVD failes on 6.x (regression) o kern/110847 scsi [ahd] Tyan U320 onboard problem with more than 3 disks 15 problems total. Non-critical problems S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/23314 scsi aic driver fails to detect Adaptec 1520B unless PnP is o kern/35234 scsi World access to /dev/pass? (for scanner) requires acce o kern/38828 scsi [feature request] DPT PM2012B/90 doesn't work o kern/44587 scsi dev/dpt/dpt.h is missing defines required for DPT_HAND o kern/76178 scsi [ahd] Problem with ahd and large SCSI Raid system o kern/96133 scsi [scsi] [patch] add scsi quirk for joyfly 128mb flash u o kern/103702 scsi [cam] [patch] ChipsBnk: Unsupported USB memory stick 7 problems total. From owner-freebsd-scsi@FreeBSD.ORG Fri May 18 13:34:21 2007 Return-Path: X-Original-To: freebsd-scsi@freebsd.org Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B357616A400 for ; Fri, 18 May 2007 13:34:21 +0000 (UTC) (envelope-from nckjagger@yahoo.com) Received: from web57515.mail.re1.yahoo.com (web57515.mail.re1.yahoo.com [66.196.100.82]) by mx1.freebsd.org (Postfix) with SMTP id 6695313C45E for ; Fri, 18 May 2007 13:34:21 +0000 (UTC) (envelope-from nckjagger@yahoo.com) Received: (qmail 69731 invoked by uid 60001); 18 May 2007 13:07:40 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:MIME-Version:Content-Type:Message-ID; b=mSghdSCbyhaupy3GG7C8c2QO4uj6z1jsw2nNpd5fJCibYKYMAwiEz5AyEV/S2fZxEuoCifYOX1/SvECdcfT2DsZjZZMUpo3MjhCh56iSmpInDPjCyvIsYQSNNl+IAv2siNZrTrfi820zTFWJr12eOMRzoAKiAhpnYSRStxvWkl4=; X-YMail-OSG: cyIvAFkVM1l1a_PZj9aB9MFGvFAEM0j5I1myuInvFXnMg51LIJreDg_rxPnnEISWXlJLLpQfoPfy2miphkpFw7mqr9WJG2YdwCjb Received: from [85.145.136.138] by web57515.mail.re1.yahoo.com via HTTP; Fri, 18 May 2007 06:07:40 PDT X-Mailer: YahooMailRC/651.23.1 YahooMailWebService/0.7.41.10 Date: Fri, 18 May 2007 06:07:40 -0700 (PDT) From: Nick Jagger To: freebsd-scsi@freebsd.org, mdodd@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=ascii Message-ID: <307487.67830.qm@web57515.mail.re1.yahoo.com> Cc: tonix@interazioni.it Subject: Re: camcontrol and IDAD0 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 May 2007 13:34:21 -0000 On Fri, 9 Sep 2005, tonix (Antonio Nati) wrote: > Finally, these controllers seems to be unusable in a production > environment with FreeBSD. I don't think anyone has really tested rebuilding a volume while the system is running. I'd be interested in the results of changing the definition of IDA_QCB_MAX from 256 to 128. See sys/dev/ida/idavar.h: #define IDA_QCB_MAX 256 -- 10 40 80 C0 00 FF FF FF FF C0 00 00 00 00 10 AA AA 03 00 00 00 08 00 ============================================ Hi Matthew, One of my DL360's with smart array controller is hanging (and resetting) about once a month with a similar error as reported by Tonix. Do you recommend changing IDA_QCB_MAX to 128? Apparently the OP had good results with this change. Why is the value still 256 in the current version of sys/dev/ida/idavar.h? What is the status of the ida patches (found in your ftp site)? Nick ____________________________________________________________________________________ 8:00? 8:25? 8:40? Find a flick in no time with the Yahoo! Search movie showtime shortcut. http://tools.search.yahoo.com/shortcuts/#news ____________________________________________________________________________________ Sucker-punch spam with award-winning protection. Try the free Yahoo! Mail Beta. http://advision.webevents.yahoo.com/mailbeta/features_spam.html From owner-freebsd-scsi@FreeBSD.ORG Fri May 18 14:27:28 2007 Return-Path: X-Original-To: freebsd-scsi@freebsd.org Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A6C9216A403 for ; Fri, 18 May 2007 14:27:28 +0000 (UTC) (envelope-from tonix@interazioni.it) Received: from mx02.interazioni.net (mx02.interazioni.net [80.94.114.204]) by mx1.freebsd.org (Postfix) with ESMTP id 82F3B13C458 for ; Fri, 18 May 2007 14:27:27 +0000 (UTC) (envelope-from tonix@interazioni.it) Received: (qmail 35340 invoked by uid 88); 18 May 2007 14:00:45 -0000 Received: by simscan 1.1.0 ppid: 35330, pid: 35334, t: 0.2013s scanners: attach: 1.1.0 clamav: 0.88.7/m: Received: from unknown (HELO IATonino.interazioni.it) (tonix@interazioni.it@151.84.104.131) by relay.interazioni.net with ESMTPA; 18 May 2007 14:00:45 -0000 X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Fri, 18 May 2007 16:00:37 +0200 To: Nick Jagger ,freebsd-scsi@freebsd.org, mdodd@freebsd.org From: "tonix (Antonio Nati)" In-Reply-To: <307487.67830.qm@web57515.mail.re1.yahoo.com> References: <307487.67830.qm@web57515.mail.re1.yahoo.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Message-Id: <20070518142727.82F3B13C458@mx1.freebsd.org> Cc: Subject: Re: camcontrol and IDAD0 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 May 2007 14:27:28 -0000 Hello, I did try one year ago. Don't have the machine now available, but I remember it worked fine and rebuild all with the new parameter. Regards, Tonino At 15.07 18/05/2007, Nick Jagger wrote: >On Fri, 9 Sep 2005, tonix (Antonio Nati) wrote: > > > Finally, these controllers seems to be unusable in a production > > > environment with FreeBSD. > > > >I don't think anyone has really tested rebuilding a volume while the > >system is running. > > > >I'd be interested in the results of changing the definition of IDA_QCB_MAX > >from 256 to 128. > > > >See sys/dev/ida/idavar.h: > > > > #define IDA_QCB_MAX 256 > > > >-- > >10 40 80 C0 00 FF FF FF FF C0 00 00 00 00 10 AA AA 03 00 00 00 08 00 > >============================================ > > > >Hi Matthew, > >One of my DL360's with smart array controller is hanging (and >resetting) about once a month with a similar error as reported by >Tonix. Do you recommend changing IDA_QCB_MAX to 128? > >Apparently the OP had good results with this change. Why is the >value still 256 in the current version of sys/dev/ida/idavar.h? > >What is the status of the ida patches (found in your ftp site)? > >Nick > > > > > > > > > > > > > > > > > > >____________________________________________________________________________________ >8:00? 8:25? 8:40? Find a flick in no time >with the Yahoo! Search movie showtime shortcut. >http://tools.search.yahoo.com/shortcuts/#news > > > > > > >____________________________________________________________________________________ >Sucker-punch spam with award-winning protection. >Try the free Yahoo! Mail Beta. >http://advision.webevents.yahoo.com/mailbeta/features_spam.html From owner-freebsd-scsi@FreeBSD.ORG Fri May 18 23:10:54 2007 Return-Path: X-Original-To: freebsd-scsi@freebsd.org Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6349916A405 for ; Fri, 18 May 2007 23:10:54 +0000 (UTC) (envelope-from mjacob@freebsd.org) Received: from ns1.feral.com (ns1.feral.com [192.67.166.1]) by mx1.freebsd.org (Postfix) with ESMTP id 24F6213C484 for ; Fri, 18 May 2007 23:10:53 +0000 (UTC) (envelope-from mjacob@freebsd.org) Received: from ns1.feral.com (localhost [127.0.0.1]) by ns1.feral.com (8.14.1/8.14.1) with ESMTP id l4INAjv9081246; Fri, 18 May 2007 16:10:53 -0700 (PDT) (envelope-from mjacob@freebsd.org) Received: from localhost (mjacob@localhost) by ns1.feral.com (8.14.1/8.14.1/Submit) with ESMTP id l4INAjAp081243; Fri, 18 May 2007 16:10:45 -0700 (PDT) (envelope-from mjacob@freebsd.org) X-Authentication-Warning: ns1.feral.com: mjacob owned process doing -bs Date: Fri, 18 May 2007 16:10:45 -0700 (PDT) From: mjacob@freebsd.org To: freebsd-scsi@freebsd.org Message-ID: <20070518153412.C75640@ns1.feral.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-performance@freebsd.org Subject: Dell/Perc5 raid/MPT SAS Integrated Raid Write Performance X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: mjacob@freebsd.org List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 May 2007 23:10:54 -0000 A lot of users have seen some very poor write performance on the MPT (LSI-Logic) driven SAS/SATA controllers, particularly those that function in Integrated Raid (mirroring) mode. Some of the reported performance issues are pretty clearly single spindle small transfer IOPS issues. For example, directory intensive and small file operations like mail server applications can do very poorly on single spindle SATA drives that are connected via a SAS channel that doesn't enable write cacheing on the SATA drive (i.e., does not flow through WCE for SCSI emulation, as the LSI-Logic *apparently* does not). Benchmarks like Postmark show pretty amazing differences when run on a PATA or native SATA based drive (1000s of ops/second) and on single Fibre Channel or SCSI drives (100s of ops/second) and can be even worse for SATA drives on a SAS controller. In these cases, there isn't much to be done- the h/w being picked doesn't match the application. However, other users have reported things which are *clearly* bad performance issues. In these cases users have reported sequential write speed to be a small fraction of read speed. That is, a single threaded read of a 10GB file will get spindle rotational speed magnitude for the disk in question (~40MB/s) but will only write at around ~6MB/s. This is clearly broken and wrong. Since I don't actually have a *lot* of MPT h/w and none that shows this write performance problem could folks do me a favor and at the next reboot get into the LSI-Logic BIOS utilities and find me all the firmware revision numbers? This might help me nail down some differences to go talk to LSI-Logic about. The overriding LSI-Logic BIOS revision is of interest, but also any of the firmware revision numbers. For example, the loaned Sunfire 4100 I have has 6.0.2.0 for the BIOS revision, but 1.04.00.00-IR for the firmware- and this system, which has two integral SAS 2.5" drives, writes at 50MB/s with them set up as an integrated mirror. -matt From owner-freebsd-scsi@FreeBSD.ORG Sat May 19 01:49:00 2007 Return-Path: X-Original-To: freebsd-scsi@freebsd.org Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4EE4D16A402; Sat, 19 May 2007 01:49:00 +0000 (UTC) (envelope-from Johannes.Kruger@nokia.com) Received: from mgw-ext14.nokia.com (smtp.nokia.com [131.228.20.173]) by mx1.freebsd.org (Postfix) with ESMTP id C90A513C448; Sat, 19 May 2007 01:48:59 +0000 (UTC) (envelope-from Johannes.Kruger@nokia.com) Received: from esebh105.NOE.Nokia.com (esebh105.ntc.nokia.com [172.21.138.211]) by mgw-ext14.nokia.com (Switch-3.2.5/Switch-3.2.5) with ESMTP id l4J1PEgo023489; Sat, 19 May 2007 04:25:37 +0300 Received: from daebh101.NOE.Nokia.com ([10.241.35.111]) by esebh105.NOE.Nokia.com with Microsoft SMTPSVC(6.0.3790.1830); Sat, 19 May 2007 04:25:20 +0300 Received: from daebe102.NOE.Nokia.com ([10.241.35.115]) by daebh101.NOE.Nokia.com with Microsoft SMTPSVC(6.0.3790.1830); Fri, 18 May 2007 20:25:17 -0500 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Date: Fri, 18 May 2007 20:25:16 -0500 Message-ID: In-Reply-To: <20070518153412.C75640@ns1.feral.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Dell/Perc5 raid/MPT SAS Integrated Raid Write Performance Thread-Index: AceZoeGVA9x/aURGQ/SNXOxe/7u4+wAECAIQ References: <20070518153412.C75640@ns1.feral.com> From: To: , X-OriginalArrivalTime: 19 May 2007 01:25:17.0736 (UTC) FILETIME=[8F1A0A80:01C799B4] X-Nokia-AV: Clean Cc: freebsd-performance@freebsd.org Subject: RE: Dell/Perc5 raid/MPT SAS Integrated Raid Write Performance X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 May 2007 01:49:00 -0000 I am using a LSI-1064E 2 boards, each with firmware images I compiled from a base firmware image concatenated with an NVDATA file that you configure yourself. I set up the ports as narrow and attach 2 SATA drives in RAID-1 configuration and I get 28-30 MBytes/sec. I compare this to the same drives running on an SATA capable IDE controller and the performance is the same. So RAID-1 does not seem to impact it ... if the volume is in sync. I have the sync rate set to about 2% default in the firmware config (NVDATA) file, and when the volume is out of sync, the performance drop to about 15 MBytes/second and jump back t full speed once in sync. Attaching SAS 2.5 inch drives I get 55-60 MBytes/sec. Using firmware revision 01.18.00.00 with MPI Version 1.5.13.0 using NVDATA file version 25. and on the other board I am experimenting with a beta phase9 (whatever that means) firmware, compiled with MPI version 1.5.14.0 and NVDATA file version 28. On both setups I removed the LSI BIOS part (mptsas.rom) after a while, since I do not need it. I configure the controller from the OS itself. This all with Matthew's driver ported to an FreeBSD 2.1.0 variant, including the NEW CAM layer. The firmware for 01.18.00.00 is freely available from their site. =20 http://www.lsi.com/storage_home/products_home/standard_product_ics/sas_i cs/lsisas1064e/index.html Use the host bus adapter firmware if you want to try it out: http://www.lsi.com/cm/License.do?url=3D/files/support/ssp/fusionmpt/sas/b= i os_firmware/SAS3041E-R_IR_10-30-06.zip&prodName=3DLSISAS3041E-R&subType=3D= Fi rmware&locale=3DEN Johan -----Original Message----- From: owner-freebsd-scsi@freebsd.org [mailto:owner-freebsd-scsi@freebsd.org]=20 Sent: Friday, May 18, 2007 7:11 PM To: freebsd-scsi@freebsd.org Cc: freebsd-performance@freebsd.org Subject: Dell/Perc5 raid/MPT SAS Integrated Raid Write Performance A lot of users have seen some very poor write performance on the MPT=20 (LSI-Logic) driven SAS/SATA controllers, particularly those that=20 function in Integrated Raid (mirroring) mode. Some of the reported performance issues are pretty clearly single=20 spindle small transfer IOPS issues. For example, directory intensive and small file operations like mail server applications can do very poorly=20 on single spindle SATA drives that are connected via a SAS channel that=20 doesn't enable write cacheing on the SATA drive (i.e., does not flow=20 through WCE for SCSI emulation, as the LSI-Logic *apparently* does not). Benchmarks like Postmark show pretty amazing differences when run on a=20 PATA or native SATA based drive (1000s of ops/second) and on single=20 Fibre Channel or SCSI drives (100s of ops/second) and can be even worse=20 for SATA drives on a SAS controller. In these cases, there isn't much to be done- the h/w being picked=20 doesn't match the application. However, other users have reported things which are *clearly* bad=20 performance issues. In these cases users have reported sequential write=20 speed to be a small fraction of read speed. That is, a single threaded=20 read of a 10GB file will get spindle rotational speed magnitude for the=20 disk in question (~40MB/s) but will only write at around ~6MB/s. This is clearly broken and wrong. Since I don't actually have a *lot* of MPT h/w and none that shows this=20 write performance problem could folks do me a favor and at the next=20 reboot get into the LSI-Logic BIOS utilities and find me all the=20 firmware revision numbers? This might help me nail down some differences to go talk to LSI-Logic about. The overriding LSI-Logic BIOS revision is of interest, but also any of=20 the firmware revision numbers. For example, the loaned Sunfire 4100 I=20 have has 6.0.2.0 for the BIOS revision, but 1.04.00.00-IR for the=20 firmware- and this system, which has two integral SAS 2.5" drives,=20 writes at 50MB/s with them set up as an integrated mirror. -matt _______________________________________________ freebsd-scsi@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-scsi To unsubscribe, send any mail to "freebsd-scsi-unsubscribe@freebsd.org"