From owner-freebsd-stable@FreeBSD.ORG Sun Nov 25 16:54:54 2007 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C7F5216A41A for ; Sun, 25 Nov 2007 16:54:54 +0000 (UTC) (envelope-from askbill@conducive.net) Received: from conducive.net (conducive.net [203.194.153.81]) by mx1.freebsd.org (Postfix) with ESMTP id 8CDE113C442 for ; Sun, 25 Nov 2007 16:54:54 +0000 (UTC) (envelope-from askbill@conducive.net) Received: from cm218-253-81-177.hkcable.com.hk ([218.253.81.177]:60633 helo=pb.local) by conducive.net with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.63 (FreeBSD)) (envelope-from ) id 1IwKkm-000H9a-Pl for freebsd-stable@freebsd.org; Sun, 25 Nov 2007 16:54:44 +0000 Message-ID: <4749A8D5.3070005@conducive.net> Date: Sun, 25 Nov 2007 16:54:45 +0000 From: =?UTF-8?B?6Z+T5a625qiZIEJpbGwgSGFja2Vy?= User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.1.2) Gecko/20070221 SeaMonkey/1.1.1 MIME-Version: 1.0 To: freebsd-stable@freebsd.org References: <47496BE1.8080206@modula.no> <47499710.1050402@tomjudge.com> <47499E65.4050206@modula.no> In-Reply-To: <47499E65.4050206@modula.no> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: SAS5IR performance issue with Dell 860 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Nov 2007 16:54:55 -0000 Espen Tagestad wrote: > Tom Judge wrote: >> Espen Tagestad wrote: >>> We recently bought 3 new Dell 860 servers with the onboard SAS5/IR >>> SATA RAID-controller. They seem to be quite well spec'ed servers with >>> management and everything - but I am experiencing av major >>> performance issue with the disc i/o. On write I get at max 7-8MB/sec, >>> while read gives a bit more (11MB/sec). I tried first with >>> 6.2-RELEASE, and then upgraded to 6.3-PRERELEASE without any better >>> results. >> >> You will also need to set the hw.mpt.enable_sata_wc sysctl in >> loader.conf. > > There isn't any hw.mpt.enable_sata_wc sysctl available on my systems. It > is 6.3-PRERELEASE, but as I recon there wasn't any *mpt* sysctls on the > latest 6.2-RELEASE either. Is it deprecated? Is there any other options? > I can see a hw.ata.wc but that one is set to 1 which I presume equals > enabled. > It is not in the man page, and neither sysctl -a nor sysctl -a -o shows it, even on 6.3-PRE. Unless you have used it. It should show up in sysctl -a -o only after first use. > Anyway - the write cache, is that something that is set on the raid > controller, or is it a buffer in the FreeBSD kernel that takes care of > the caching? Both and more. The BSD buffers can generally be left alone to 'do the right thing'. Buffers on the drive itself, likewise. Mucking about with either is likely to do more harm than good overall unless your only goal is fast (looking) benchmarks. In which case attach an old Zip drive, remove the media, and watch truly astonishing benchmarks ... as the system does r/w to/from the in-RAM cache. Only. ;-) Buffers 'managed by' the controller, whether physically onboard or an allocated chunk of system RAM depend on the needs of your application. EX: A PostgreSQL DB is 'transactionally aware' and has its own Write Ahead Logging and commit/rollback tools. It is generally fast AND safe with Softupdates OFF for the mount-point or device where it stores its data and controller cache set to 'write through' rather than 'write back'. A differently prioritized application might need either write-back caching and Softupdates. One, the other, neither, or both. It is ever a tradeoff between recoverability and (apparent) speed, and too much tuning can be counterproductive for the more general cases. I say 'apparent' because at the end of the day, the only writes that do not have to be made at all are those 'known to have' gone stale while still in the OS buffers. Or - better yet - while in L1/L2/L3 CPU cache... ;-) > As the commit note you sent me said - to ensure absolutely > best data integrity the write cache should be left switched off. But > write performance of 7-8MB/sec is just too low for that - is the > controller /sata drives really that slow? > Depends very much on what (else) they and their drives are doing. Drive head positioning is the killer that channel speed can't do much about. If the controller and drives are dedicated solely to one app or file transfer, do no logging, have no system tools or such on the same platter(s) that might need interruptions to their measured task, are able to do r/w in damn-near-slew-mode, they should be much faster. If there are any system-relative mounts on the same spindle(s) (/var/log comes first to mind) - then probably not. Heads have to go elsewhere and back. Real-world *NIX boxen are rarely in the position of concentrating on just one I/O task at a time. ...and you still haven't said what you are using to derive the numbers, what sort of drives are at the end of the cable, or if this is an application or test suite, and/or if the r/w is to drives that do NOT do anything else...... ?? Bill