From owner-freebsd-current@FreeBSD.ORG Tue Dec 8 00:31:59 2009 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from [127.0.0.1] (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by hub.freebsd.org (Postfix) with ESMTP id CE6AE106566B; Tue, 8 Dec 2009 00:31:59 +0000 (UTC) (envelope-from jkim@FreeBSD.org) From: Jung-uk Kim To: freebsd-current@FreeBSD.org Date: Mon, 7 Dec 2009 19:31:43 -0500 User-Agent: KMail/1.6.2 References: <3c0b01820912071342u1c722b2clf9c8413e40097279@mail.gmail.com> <3c0b01820912071430o545e0ae4u45cb3b658f48c306@mail.gmail.com> In-Reply-To: <3c0b01820912071430o545e0ae4u45cb3b658f48c306@mail.gmail.com> MIME-Version: 1.0 Content-Disposition: inline Content-Type: Multipart/Mixed; boundary="Boundary-00=_x5ZHLV0Wa6qggXx" Message-Id: <200912071931.46002.jkim@FreeBSD.org> Cc: Alexander Sack , scottl@FreeBSD.org, emaste@FreeBSD.org Subject: Re: aac(4) resource FIB starvation on BUS scan revisited X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2009 00:32:00 -0000 --Boundary-00=_x5ZHLV0Wa6qggXx Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 8bit Content-Disposition: inline On Monday 07 December 2009 05:30 pm, Alexander Sack wrote: > On Mon, Dec 7, 2009 at 4:42 PM, Alexander Sack wrote: > > Folks: > > > > I posted a similar thread on freebsd-scsi only to realize that > > scottl had fixed my first issue during some MP CAM cleanup with > > respect to a race during resource allocation issues on a later > > version of the driver we are using (I believe we did the same > > thing to resolve a lock issue on bootup). > > > > However on my RELENG_8 box with (2) Adaptec 5085s connected to > > some JBODs (9TB each) I still have a FIB starvation issue during > > the LUN scan: > > > > The number of FIBs allocated to this card is 512 (older cards are > > 256).  The max_target per bus is 287.  On a six channel > > controller with a BUS scan done in parallel I see a lot of this: > > > > ... > > (probe501:aacp1:0:214:0): Request Requeued > > (probe501:aacp1:0:214:0): Retrying Command > > (probe520:aacp1:0:233:0): Request Requeued > > (probe520:aacp1:0:233:0): Retrying Command > > (probe528:aacp1:0:241:0): Request Requeued > > (probe528:aacp1:0:241:0): Retrying Command > > (probe540:aacp1:0:253:0): Request Requeued > > (probe540:aacp1:0:253:0): Retrying Command > > (probe541:aacp1:0:254:0): Request Requeued > > (probe541:aacp1:0:254:0): Retrying Command > > .... > > > > I think the driver is much happier with the following attached > > patch (with dmesg). > > Patch again but this time not base-64 encoded: [SNIP!] I want it to be little conservative here, i.e., pre-allocating half of max_fibs. Will the attached patch work for you? Thanks, Jung-uk Kim --Boundary-00=_x5ZHLV0Wa6qggXx Content-Type: text/plain; charset="utf-8"; name="aac.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="aac.diff" Index: sys/dev/aac/aacvar.h =================================================================== --- sys/dev/aac/aacvar.h (revision 200237) +++ sys/dev/aac/aacvar.h (working copy) @@ -57,13 +57,6 @@ #define AAC_ADAPTER_FIBS 8 /* - * FIBs are allocated in page-size chunks and can grow up to the 512 - * limit imposed by the hardware. - */ -#define AAC_PREALLOCATE_FIBS 128 -#define AAC_NUM_MGT_FIB 8 - -/* * The controller reports status events in AIFs. We hang on to a number of * these in order to pass them out to user-space management tools. */ Index: sys/dev/aac/aac.c =================================================================== --- sys/dev/aac/aac.c (revision 200237) +++ sys/dev/aac/aac.c (working copy) @@ -604,7 +604,7 @@ aac_alloc(struct aac_softc *sc) TAILQ_INIT(&sc->aac_fibmap_tqh); sc->aac_commands = malloc(sc->aac_max_fibs * sizeof(struct aac_command), M_AACBUF, M_WAITOK|M_ZERO); - while (sc->total_fibs < AAC_PREALLOCATE_FIBS) { + while (sc->total_fibs < sc->aac_max_fibs / 2) { if (aac_alloc_commands(sc) != 0) break; } Index: sys/dev/aac/aac_cam.c =================================================================== --- sys/dev/aac/aac_cam.c (revision 200237) +++ sys/dev/aac/aac_cam.c (working copy) @@ -260,8 +260,11 @@ aac_cam_action(struct cam_sim *sim, union ccb *ccb cpi->hba_inquiry = PI_WIDE_16; cpi->target_sprt = 0; - /* Resetting via the passthrough causes problems. */ - cpi->hba_misc = PIM_NOBUSRESET; + /* + * Resetting via the passthrough or parallel bus scan + * causes problems. + */ + cpi->hba_misc = PIM_NOBUSRESET | PIM_SEQSCAN; cpi->hba_eng_cnt = 0; cpi->max_target = camsc->inf->TargetsPerBus; cpi->max_lun = 8; /* Per the controller spec */ --Boundary-00=_x5ZHLV0Wa6qggXx--