From owner-freebsd-scsi@FreeBSD.ORG Wed Aug 14 15:16:08 2013 Return-Path: Delivered-To: FreeBSD-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id E4B2A756; Wed, 14 Aug 2013 15:16:08 +0000 (UTC) (envelope-from prvs=19381d5a45=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5FC162185; Wed, 14 Aug 2013 15:16:08 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50005492799.msg; Wed, 14 Aug 2013 16:16:04 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Wed, 14 Aug 2013 16:16:04 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=19381d5a45=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: <46DE157BA89B4C17B95BE5E738D0B2AF@multiplay.co.uk> From: "Steven Hartland" To: "Doug Ambrisko" , References: <1373822621.1431.5.camel@localhost> <1376448416.1439.7.camel@localhost> <20130814150331.GB34825@ambrisko.com> Subject: Re: Dell H310, JBOD mode "hard error" Date: Wed, 14 Aug 2013 16:16:23 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: FreeBSD-scsi@freebsd.org X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Aug 2013 15:16:09 -0000 ----- Original Message ----- From: "Doug Ambrisko" To: Cc: Sent: Wednesday, August 14, 2013 4:03 PM Subject: Re: Dell H310, JBOD mode "hard error" > On Tue, Aug 13, 2013 at 07:46:56PM -0700, Sean Bruno wrote: > | On Sun, 2013-07-14 at 10:23 -0700, Sean Bruno wrote: > | > Not sure what to make of this. I've tested a lot of svn revisions of > | > the thunderbolt code, but nothing looks obvious. > | > > | > When I use a single drive in "SYSPD" mode on a Dell H310 (falcon or > | > skinny drake) I get a /dev/mfisyspd0 device. The JBOD mode *seems* to > | > work just fine as long as I don't do multiple things at once to it, e.g. > | > single user fsck works, but multiuser things die. > | > > | > I get a failure case that emits errors such as: > | > > | > g_vfs_done():error 27 in callback > | > mfisyspd0p2[READ(offset=7176192, length=425984)]mfisyspd0: hard error > | > error = 5 > | > cmd=read 15360-16383 > | > error 27 in callback > | > g_vfs_done():mfisyspd0: hard error mfisyspd0p2[READ(offset=7602176, > | > length=524288)]cmd=read error = 5 > | > 16384-17407 > | > error 27 in callback > | > g_vfs_done():mfisyspd0: hard error mfisyspd0p2[READ(offset=8126464, > | > length=524288)]cmd=read error = 5 > | > 14560-15359 > | > error 27 in callback > | > g_vfs_done():mfisyspd0: hard error mfisyspd0p2[READ(offset=7192576, > | > length=409600)]cmd=read error = 5 > | > 15360-16383 > | > > | > > | > Sean > | > | Ah, I see something that Yahoo! does that FreeBSD does not finally. We > | tune MAXPHYS *up* to (512 * 1024) because of performance and available > | memory. > | > | mfi(4) set's its own (MFI_MAXPHYS) to (128 * 1024) instead of using the > | value from sys/param.h (btw, I don't quite get why, but whatever). > | > | Without a min() check in mfi_syspd.c that mimics the one in mfi_disk.c, > | Yahoo code falls over in "SYSPD" mode (mfi(4) real jbod mode). > | > | Patch: > | > | Index: mfi_syspd.c > | =================================================================== > | --- mfi_syspd.c (revision 254313) > | +++ mfi_syspd.c (working copy) > | @@ -128,7 +128,9 @@ > | sc->pd_disk->d_drv1 = sc; > | sc->pd_disk->d_maxsize = sc->pd_controller->mfi_max_io * secsize; > | sc->pd_disk->d_name = "mfisyspd"; > | - sc->pd_disk->d_open = mfi_syspd_open; > | + sc->pd_disk->d_maxsize = min(sc->pd_controller->mfi_max_io * secsize, > | + (sc->pd_controller->mfi_max_sge - 1) * PAGE_SIZE); > | + > | sc->pd_disk->d_close = mfi_syspd_close; > | sc->pd_disk->d_strategy = mfi_syspd_strategy; > | sc->pd_disk->d_dump = mfi_syspd_dump; > > > That change for d_maxsize looks okay but do you really want to get > rid of d_open? I assume this is a cut-n-paste type error and the patch > (hand editted) should be: > > Index: mfi_syspd.c > =================================================================== > --- mfi_syspd.c (revision 254313) > +++ mfi_syspd.c (working copy) > @@ -128,7 +128,8 @@ > sc->pd_disk->d_drv1 = sc; > - sc->pd_disk->d_maxsize = sc->pd_controller->mfi_max_io * secsize; > + sc->pd_disk->d_maxsize = min(sc->pd_controller->mfi_max_io * secsize, > + (sc->pd_controller->mfi_max_sge - 1) * PAGE_SIZE); > sc->pd_disk->d_name = "mfisyspd"; > sc->pd_disk->d_open = mfi_syspd_open; > sc->pd_disk->d_close = mfi_syspd_close; > sc->pd_disk->d_strategy = mfi_syspd_strategy; > sc->pd_disk->d_dump = mfi_syspd_dump; > > BTW, has mfiutil been updated to create real JBODs versus the RAID per > drive? I know someone was talking about doing that. A note on implementing > it, it also requires JBOD to be enabled in the controller. I'm not sure > if all controllers support it. I forget when I was playing with it. I've > always wondering if we should change the name for the syspd disk node > but I left it for compatibility with LSI. We could do an alias. It is > good in away that it doesn't create /dev/mfid* nodes so that it is easier > to track bugs. Some do support native JBOD, some require an additional controller level setting, and some simply don't support it and I'm not aware of an easy way to determine which is the case I'm afraid. Last time I looked mfiutil used RAID not native JBOD for JBOD's Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk.