From owner-freebsd-performance@FreeBSD.ORG Mon Sep 5 00:18:39 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4C7D616A41F for ; Mon, 5 Sep 2005 00:18:39 +0000 (GMT) (envelope-from jeff.tchang@gmail.com) Received: from rproxy.gmail.com (rproxy.gmail.com [64.233.170.205]) by mx1.FreeBSD.org (Postfix) with ESMTP id BCF4043D45 for ; Mon, 5 Sep 2005 00:18:38 +0000 (GMT) (envelope-from jeff.tchang@gmail.com) Received: by rproxy.gmail.com with SMTP id r35so763494rna for ; Sun, 04 Sep 2005 17:18:36 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:mime-version:content-type; b=OpDzRTgOo32By6C3UTHJDRimm27T9BDGu0uZLmNvVAbhx8/NdS0QqpP0BSmfM1gB5EeZizIuXIDtLFsbkL3Nr39GzjhbqZFe7EZSI0aikOiwqGDtWBgLAeZCp0GuD6LFj4RkNlzDzDC/QsmCbDE6uAHpl8Xl++A7LKtF2QI0SdI= Received: by 10.38.10.54 with SMTP id 54mr30035rnj; Sun, 04 Sep 2005 17:18:35 -0700 (PDT) Received: by 10.39.1.43 with HTTP; Sun, 4 Sep 2005 17:18:35 -0700 (PDT) Message-ID: <63f9d26505090417183dff415e@mail.gmail.com> Date: Sun, 4 Sep 2005 20:18:35 -0400 From: Jeff Tchang To: freebsd-performance@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: 3Ware 7500-4 Slow X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Sep 2005 00:18:39 -0000 I have a 3Ware 7500-4 card. I am experiencing some sluggishness with the=20 RAID5 implementation. It has been running in a stable state for quite some= =20 time, however, I have always wondered if there was something I was not doin= g=20 because it is very slow. System Specs: pharos# dmesg | grep twe twe0: <3ware Storage Controller driver ver. 1.40.01.001 >=20 port 0xccd0-0xccdf mem 0xfe000000-0xfe7fffff,0xfe800400-0xfe80040f irq 2 at= =20 device 16.0 on pci0 twe0: 4 ports, Firmware FE7X 1.05.00.068 , BIOS BE7X=20 1.08.00.048 twed0: on twe0 twed0: 152636MB (312598784 sectors) Mounting root from ufs:/dev/twed0s1a uname -a FreeBSD 4.10-RELEASE FreeBSD 4.10-RELEASE #4: Sat Sep 3 15:53:44 EDT 2005= =20 i386 3ware CLI> info List of controllers ------------------- Controller 0: 7500-4 (4) 3ware CLI> info c0 Controller: c0 ------------- Driver: 1.40.01.001 Model: 7500-4 FW: FE7X 1.05.00.068 BIOS: BE7X 1.08.00.048 Monitor: ME7X 1.01.00.040 Serial #:=20 PCB: Rev3 PCHIP: 1.30-33 ACHIP: 3.20 # of units: 1 Unit 0: RAID 5 149.05 GB ( 312598784 blocks): OK # of ports: 4 Port 0: ST380020A 74.53 GB (156301488 blocks): OK(unit 0) Port 1: ST380020A 74.53 GB (156301488 blocks): OK(unit 0) Port 2: ST380020A 74.53 GB (156301488 blocks): OK(unit 0) 3ware CLI> info c0 u0 Unit /c0/u0 --------------------- Status: OK Unit Type: RAID 5 Stripe Size: 64k Size: 149.05 GB (312598784 blocks) # of subunits: 3 Subunit 0: CBOD: OK Physical Port: 0 Logical Port: 0 Subunit 1: CBOD: OK Physical Port: 1 Logical Port: 1 Subunit 2: CBOD: OK Physical Port: 2 Logical Port: 2 When I run systat -vmstat 1, I continually see twed0 running at 100% busy.= =20 In addition KB/t is between 1 and 10. tps is about 50. The dmesg says that the device is on irq2. The vmstat shows the interrupt a= s=20 mux irq2 and is about 300 to 500 calls. As far as I know, I am using the=20 most up to date firmware for the card. The computer itself is a dual 600 Mh= z=20 PIII (Dell). When I attempt to write many small files or remove a directory is when the= =20 slowness kicks in. Is this just something due to Raid5? Here is the output= =20 of Bonnie++: Version 1.93c ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP pharos. 300M 53 94 16394 24 7163 14 121 95 35991 35 249.8 12 Latency 261ms 400ms 485ms 93237us 26252us 12216ms Version 1.93c ------Sequential Create------ --------Random Create-------- pharos -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 3835 45 28846 93 8255 85 4249 56 26604 84 4835 52 Latency 43557us 3422us 8544us 169ms 39158us 6936us From owner-freebsd-performance@FreeBSD.ORG Mon Sep 5 00:37:30 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8A5DD16A41F for ; Mon, 5 Sep 2005 00:37:30 +0000 (GMT) (envelope-from killing@multiplay.co.uk) Received: from multiplay.co.uk (core6.multiplay.co.uk [85.236.96.23]) by mx1.FreeBSD.org (Postfix) with ESMTP id D4A8443D48 for ; Mon, 5 Sep 2005 00:37:29 +0000 (GMT) (envelope-from killing@multiplay.co.uk) Received: from vader ([212.135.219.179]) by multiplay.co.uk (multiplay.co.uk [85.236.96.23]) (MDaemon.PRO.v8.1.1.R) with ESMTP id md50001793759.msg for ; Mon, 05 Sep 2005 01:36:38 +0100 Message-ID: <02bc01c5b1b1$dc623ac0$b3db87d4@multiplay.co.uk> From: "Steven Hartland" To: "Jeff Tchang" , References: <63f9d26505090417183dff415e@mail.gmail.com> Date: Mon, 5 Sep 2005 01:36:12 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2670 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2670 X-Spam-Processed: multiplay.co.uk, Mon, 05 Sep 2005 01:36:38 +0100 (not processed: message from valid local sender) X-MDRemoteIP: 212.135.219.179 X-Return-Path: killing@multiplay.co.uk X-MDaemon-Deliver-To: freebsd-performance@freebsd.org X-MDAV-Processed: multiplay.co.uk, Mon, 05 Sep 2005 01:36:38 +0100 Cc: Subject: Re: 3Ware 7500-4 Slow X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Sep 2005 00:37:30 -0000 >From what I've seen with such a slow machine and only 3 disks I doubt you would get good performance. Steve ----- Original Message ----- From: "Jeff Tchang" I have a 3Ware 7500-4 card. I am experiencing some sluggishness with the RAID5 implementation. It has been running in a stable state for quite some time, however, I have always wondered if there was something I was not doing because it is very slow. ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone (023) 8024 3137 or return the E.mail to postmaster@multiplay.co.uk. From owner-freebsd-performance@FreeBSD.ORG Mon Sep 5 15:45:59 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2FF2716A41F for ; Mon, 5 Sep 2005 15:45:59 +0000 (GMT) (envelope-from cswiger@mac.com) Received: from pi.codefab.com (pi.codefab.com [199.103.21.227]) by mx1.FreeBSD.org (Postfix) with ESMTP id CAF9543D48 for ; Mon, 5 Sep 2005 15:45:58 +0000 (GMT) (envelope-from cswiger@mac.com) Received: from localhost (localhost [127.0.0.1]) by pi.codefab.com (Postfix) with ESMTP id E7CBB5E96; Mon, 5 Sep 2005 11:45:57 -0400 (EDT) Received: from pi.codefab.com ([127.0.0.1]) by localhost (pi.codefab.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 89488-04; Mon, 5 Sep 2005 11:45:56 -0400 (EDT) Received: from [192.168.1.3] (pool-68-161-79-217.ny325.east.verizon.net [68.161.79.217]) by pi.codefab.com (Postfix) with ESMTP id 579485E18; Mon, 5 Sep 2005 11:45:56 -0400 (EDT) Message-ID: <431C683B.1080803@mac.com> Date: Mon, 05 Sep 2005 11:46:03 -0400 From: Chuck Swiger Organization: The Courts of Chaos User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.11) Gecko/20050801 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Jeff Tchang References: <63f9d26505090417183dff415e@mail.gmail.com> In-Reply-To: <63f9d26505090417183dff415e@mail.gmail.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: amavisd-new at codefab.com Cc: freebsd-performance@freebsd.org Subject: Re: 3Ware 7500-4 Slow X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Sep 2005 15:45:59 -0000 Jeff Tchang wrote: [ ... ] > When I attempt to write many small files or remove a directory is when the > slowness kicks in. Is this just something due to Raid5? Here is the output > of Bonnie++: Small writes are pretty much the worst-case scenario for RAID-5, and it's normal to see a very significant performance drop-- by up to an order of magnitude-- from the performance of a bare drive. Good RAID controllers with lots of RAM can reduce the performance hit to perhaps less than a factor of 2. Choosing RAID-1 rather than RAID-5 for situations which are not read-only or read-mostly is probably a good idea. -- -Chuck