From owner-freebsd-current@FreeBSD.ORG Sun Mar 21 16:32:43 2010 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0A6E1106566C; Sun, 21 Mar 2010 16:32:43 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id 9CDF78FC22; Sun, 21 Mar 2010 16:32:42 +0000 (UTC) Received: from phobos.samsco.home (phobos.samsco.home [192.168.254.11]) (authenticated bits=0) by pooker.samsco.org (8.14.3/8.14.3) with ESMTP id o2LGWdqu036885; Sun, 21 Mar 2010 10:32:39 -0600 (MDT) (envelope-from scottl@samsco.org) Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=us-ascii From: Scott Long In-Reply-To: <4BA52179.9030903@FreeBSD.org> Date: Sun, 21 Mar 2010 10:32:39 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <39C5864C-8A6B-4137-8743-D7B9F30F5939@samsco.org> References: <4BA4E7A9.3070502@FreeBSD.org> <201003201753.o2KHrH5x003946@apollo.backplane.com> <891E2580-8DE3-4B82-81C4-F2C07735A854@samsco.org> <4BA52179.9030903@FreeBSD.org> To: Alexander Motin X-Mailer: Apple Mail (2.1077) X-Spam-Status: No, score=-1.0 required=3.8 tests=ALL_TRUSTED autolearn=unavailable version=3.3.0 X-Spam-Checker-Version: SpamAssassin 3.3.0 (2010-01-18) on pooker.samsco.org Cc: FreeBSD-Current , freebsd-arch@freebsd.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Mar 2010 16:32:43 -0000 On Mar 20, 2010, at 1:26 PM, Alexander Motin wrote: > Scott Long wrote: >> On Mar 20, 2010, at 11:53 AM, Matthew Dillon wrote: >>> Diminishing returns get hit pretty quickly with larger MAXPHYS = values. >>> As long as the I/O can be pipelined the reduced transaction rate >>> becomes less interesting when the transaction rate is less than a >>> certain level. Off the cuff I'd say 2000 tps is a good basis for >>> considering whether it is an issue or not. 256K is actually quite >>> a reasonable value. Even 128K is reasonable. >>=20 >> I agree completely. I did quite a bit of testing on this in 2008 and = 2009. >> I even added some hooks into CAM to support this, and I thought that = I had >> discussed this extensively with Alexander at the time. Guess it was = yet another >> wasted conversation with him =3D-( I'll repeat it here for the = record. >=20 > AFAIR at that time you've agreed that 256K gives improvements, and 64K > of DFLTPHYS limiting most SCSI SIMs is too small. That's why you've > implemented that hooks in CAM. I have not forgot that conversation = (pity > that it quietly died for SCSI SIMs). I agree that too high value could > be just a waste of resources. As you may see I haven't blindly = committed > it, but asked public opinion. If you think 256K is OK - let it be = 256K. > If you think that 256K needed only for media servers - OK, but lets = make > it usable there. >=20 I think that somewhere in the range of 128-512k is appropriate for a = given platform. Maybe big-iron gets 512k and notebooks and embedded systems get 128k? = It's partially a platform architecture issue, and partially a platform = application issue. Ultimately, it should be possible to have up to 1M, and maybe even more. = I don't know how best to make that selectable, or whether it should just be the = default. >> Besides the nswbuf sizing problem, there is a real problem that a lot = of drivers >> have incorrectly assumed over the years that MAXPHYS and DFLTPHYS are >> particular values, and they've sized their data structures = accordingly. Before >> these values are changed, an audit needs to be done OF EVERY SINGLE >> STORAGE DRIVER. No exceptions. This isn't a case of changing MAXHYS >> in the ata driver, testing that your machine boots, and then = committing the change >> to source control. Some drivers will have non-obvious restrictions = based on >> the number of SG elements allowed in a particular command format. = MPT >> comes to mind (its multi message SG code seems to be broken when I = tried >> testing large MAXPHYS on it), but I bet that there are others. >=20 > As you should remember, we have made it in such way, that all = unchecked > drivers keep using DFLTPHYS, which is not going to be changed ever. So > there is no problem. I would more worry about non-CAM storages and = above > stuff, like some rare GEOM classes. And that's why I say that everything needs to be audited. Are there CAM = drivers that default to being silent on cpi->maxio, but still look at DFLTPHYS = and MAXPHYS? Are there non-CAM drivers that look at MAXPHYS, or that silently assume = that MAXPHYS will never be more than 128k? Scott