From owner-freebsd-performance@FreeBSD.ORG Sun Mar 4 16:51:02 2007 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5639C16A401 for ; Sun, 4 Mar 2007 16:51:02 +0000 (UTC) (envelope-from ohartman@mail.zedat.fu-berlin.de) Received: from outpost1.zedat.fu-berlin.de (outpost1.zedat.fu-berlin.de [130.133.4.66]) by mx1.freebsd.org (Postfix) with ESMTP id 6A98213C4B2 for ; Sun, 4 Mar 2007 16:51:01 +0000 (UTC) (envelope-from ohartman@mail.zedat.fu-berlin.de) Received: from inpost2.zedat.fu-berlin.de ([130.133.4.69]) by outpost1.zedat.fu-berlin.de (Exim 4.66) with esmtp (envelope-from ) id <1HNtdW-0006Bc-RO>; Sun, 04 Mar 2007 17:32:38 +0100 Received: from e178015054.adsl.alicedsl.de ([85.178.15.54] helo=[192.168.1.128]) by inpost2.zedat.fu-berlin.de (Exim 4.66) with esmtpsa (envelope-from ) id <1HNtdW-0004sL-DE>; Sun, 04 Mar 2007 17:32:38 +0100 Message-ID: <45EAF4E7.6040708@mail.zedat.fu-berlin.de> Date: Sun, 04 Mar 2007 17:33:43 +0100 From: "O. Hartmann" User-Agent: Thunderbird 1.5.0.9 (X11/20070211) MIME-Version: 1.0 References: <45E7F09B.7070005@zedat.fu-berlin.de> <200703021929.l22JTdrU091774@lava.sentex.ca> In-Reply-To: <200703021929.l22JTdrU091774@lava.sentex.ca> X-Enigmail-Version: 0.94.1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Originating-IP: 85.178.15.54 X-Mailman-Approved-At: Sun, 04 Mar 2007 20:40:58 +0000 Cc: freebsd-performance@freebsd.org, freebsd-questions@freebsd.org Subject: Re: (S)ATA performance in FBSD 6.2/7.0 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Mar 2007 16:51:02 -0000 Mike Tancsa wrote: > At 04:38 AM 3/2/2007, O. Hartmann wrote: >> The last days I tried to figure out why some of my lab's FreeBSD >> boxes and also mine at home seem to be outperformed by some Linux >> setups around here and I saw something interesting. >> >> On my lab's FreeBSD 6.2/i386 box (ASUS P4P800, ICH5 with two SATA 150 >> ports, two SATA 300 drives attached) I copied big files (~ 5GB) from >> one drive to > > > Something strange about your setup I would say. I just tried on a > Segate SATA drive off an ICH5 chipset (plain old P IV 2.4Ghz). Do you > have an option in your BIOS for "native mode" or compatibility mode > for the SATA controller ? If so, try toggling that to native SATA mode > > [ns4]% iostat -c 1000 > tty ad4 twed0 cpu > tin tout KB/t tps MB/s KB/t tps MB/s us ni sy in id > 2 447 4.91 0 0.00 23.77 40 0.92 20 0 6 0 74 > 4 307 0.00 0 0.00 12.61 14 0.17 0 0 0 0 100 > 1 183 0.00 0 0.00 14.50 4 0.06 0 0 0 0 100 > 1 63 128.00 47 5.82 0.00 0 0.00 7 0 7 0 86 > 0 182 128.00 534 66.70 15.25 8 0.12 0 0 15 8 77 > 0 60 128.00 553 69.13 2.00 2 0.00 0 0 8 8 85 > 0 182 128.00 537 67.14 14.50 4 0.06 15 0 31 15 38 > 0 60 128.00 553 69.06 0.00 0 0.00 54 0 0 8 38 > 0 60 128.00 538 67.21 0.00 0 0.00 23 0 0 8 69 > 1 301 128.00 495 61.88 12.18 22 0.26 0 0 8 0 92 > > > [ns4]# dd if=/dev/ad4 of=/dev/null bs=1024k > ^C410+0 records in > 410+0 records out > 429916160 bytes transferred in 6.089321 secs (70601659 bytes/sec) > [ns4]# > > > [ns4]# atacontrol cap ad4 > > Protocol Serial ATA II > device model ST3400833NS > serial number 5NF25DTG > firmware revision 3.AEH > cylinders 16383 > heads 16 > sectors/track 63 > lba supported 268435455 sectors > lba48 supported 781422768 sectors > dma supported > overlap not supported > > Feature Support Enable Value Vendor > write cache yes yes > read ahead yes yes > Native Command Queuing (NCQ) yes - 31/0x1F > Tagged Command Queuing (TCQ) no no 31/0x1F > SMART yes yes > microcode download yes yes > security yes no > power management yes yes > advanced power management no no 65278/0xFEFE > automatic acoustic management no no 0/0x00 254/0xFE > [ns4]# > Sorry for waiting so long, but weekend and a lot of work prevented me from answering earlier. All right, I checked on three boxes the BIOS settings and ensured no legacy settings on any SATA device. In general, I realized using 'dd' results in much better drive-to-drive bandwith than doing just a 'cp'. As many times recommended herein, I used bs=128k, but look for yourself on the results. Box A, FreeBSD 7.0-CURRENT/AMD64: FreeBSD foo.org 7.0-CURRENT FreeBSD 7.0-CURRENT #38: Sun Mar 4 12:45:42 CET 2007 root@foor.org:/usr/obj/usr/src/sys/FOO amd64 Relevant harware: Copyright (c) 1992-2007 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 7.0-CURRENT #38: Sun Mar 4 12:45:42 CET 2007 root@thor.walstatt.dyndns.org:/usr/obj/usr/src/sys/THOR ACPI APIC Table: Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: AMD Athlon(tm) 64 Processor 3500+ (2210.07-MHz K8-class CPU) Origin = "AuthenticAMD" Id = 0x10ff0 Stepping = 0 Features=0x78bfbff AMD Features=0xe2500800 AMD Features2=0x1 usable memory = 2136956928 (2037 MB) avail memory = 2062622720 (1967 MB) ioapic0 irqs 0-23 on motherboard kbd1 at kbdmux0 acpi0: on motherboard acpi0: [ITHREAD] acpi0: Power Button (fixed) Timecounter "ACPI-safe" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x508-0x50b on acpi0 cpu0: on acpi0 [...] atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xffa0-0xffaf at device 15.0 on pci0 ata0: on atapci0 ata0: [ITHREAD] ata1: on atapci0 ata1: [ITHREAD] atapci1: port 0xb480-0xb487,0xb400-0xb403,0xb080-0xb087,0xb000-0xb003,0xac00-0xac0f mem 0xddcf9000-0xddcf9fff irq 23 at device 16.0 on pci0 atapci1: [ITHREAD] ata2: on atapci1 ata2: [ITHREAD] ata3: on atapci1 ata3: [ITHREAD] atapci2: port 0xa880-0xa887,0xa800-0xa803,0xa480-0xa487,0xa400-0xa403,0xa080-0xa08f mem 0xddcf8000-0xddcf8fff irq 20 at device 17.0 on pci0 atapci2: [ITHREAD] ata4: on atapci2 ata4: [ITHREAD] ata5: on atapci2 ata5: [ITHREAD] [...] thor# atacontrol cap ad4 Protocol Serial ATA II device model HDT722525DLA380 serial number VDK41BT4EJJW3K firmware revision V44OA96A cylinders 16383 heads 16 sectors/track 63 lba supported 268435455 sectors lba48 supported 488397168 sectors dma supported overlap not supported Feature Support Enable Value Vendor write cache yes yes read ahead yes yes Native Command Queuing (NCQ) yes - 31/0x1F Tagged Command Queuing (TCQ) no no 31/0x1F SMART yes yes microcode download yes yes security yes no power management yes yes advanced power management yes no 0/0x00 automatic acoustic management yes no 254/0xFE 128/0x80 thor# atacontrol cap ad6 Protocol Serial ATA II device model HDT722525DLA380 serial number VDK41BT4EJ58WK firmware revision V44OA96A cylinders 16383 heads 16 sectors/track 63 lba supported 268435455 sectors lba48 supported 488397168 sectors dma supported overlap not supported Feature Support Enable Value Vendor write cache yes yes read ahead yes yes Native Command Queuing (NCQ) yes - 31/0x1F Tagged Command Queuing (TCQ) no no 31/0x1F SMART yes yes microcode download yes yes security yes no power management yes yes advanced power management yes no 0/0x00 automatic acoustic management yes no 254/0xFE 128/0x80 thor# atacontrol cap ad8 Protocol Serial ATA II device model SAMSUNG SP2004C serial number S07GJ10Y304470 firmware revision VM100-31 cylinders 16383 heads 16 sectors/track 63 lba supported 268435455 sectors lba48 supported 390721968 sectors dma supported overlap not supported Feature Support Enable Value Vendor write cache yes yes read ahead yes yes Native Command Queuing (NCQ) yes - 31/0x1F Tagged Command Queuing (TCQ) no no 31/0x1F SMART yes yes microcode download yes yes security yes no power management yes yes advanced power management no no 0/0x00 automatic acoustic management yes no 0/0x00 254/0xFE The dmesg portion reflecting the drive configuration on the above mentioned box: acd0: DVDR at ata0-master UDMA33 ad4: 238475MB at ata2-master SATA300 ad6: 238475MB at ata3-master SATA300 ad8: 190782MB at ata4-master SATA300 ar0: 476950MB status: READY ar0: disk0 READY using ad6 at ata3-master ar0: disk1 READY using ad4 at ata2-master Below some test results using dd: 1) dd if=/dev/ad8 of=/dev/null (/dev/null is on ar0) thor# iostat -c 25 tty ad4 ad6 ad8 cpu tin tout KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy in id 71 35 2.57 0 0.00 2.61 0 0.00 22.55 249 5.47 3 0 1 0 96 1 231 0.00 0 0.00 0.00 0 0.00 0.50 7535 3.68 2 0 30 6 62 287 78 0.00 0 0.00 0.00 0 0.00 0.50 7474 3.65 0 0 30 4 65 351 78 0.00 0 0.00 0.00 0 0.00 0.50 7475 3.65 0 0 20 6 74 519 78 0.00 0 0.00 0.00 0 0.00 0.50 7484 3.65 2 0 20 6 72 671 78 0.00 0 0.00 0.00 0 0.00 0.50 7535 3.68 0 0 17 4 79 559 78 0.00 0 0.00 0.00 0 0.00 0.50 7470 3.65 0 0 24 2 74 495 78 0.00 0 0.00 0.00 0 0.00 0.50 7469 3.65 2 0 25 2 72 511 78 0.00 0 0.00 0.00 0 0.00 0.50 7504 3.66 0 0 20 3 77 144 78 0.00 0 0.00 0.00 0 0.00 0.50 7521 3.67 0 0 27 4 69 0 78 0.00 0 0.00 0.00 0 0.00 0.50 7474 3.65 0 0 26 6 68 0 78 0.00 0 0.00 0.00 0 0.00 0.50 7476 3.65 0 0 24 8 68 136 78 0.00 0 0.00 0.00 0 0.00 0.50 7506 3.66 0 0 27 5 68 40 78 0.00 0 0.00 0.00 0 0.00 0.50 7513 3.67 0 0 24 4 71 567 78 0.00 0 0.00 0.00 0 0.00 0.50 7465 3.64 2 0 27 4 68 455 78 0.00 0 0.00 0.00 0 0.00 0.50 7482 3.65 1 0 28 5 65 655 78 0.00 0 0.00 0.00 0 0.00 0.50 7543 3.68 2 0 28 8 61 208 78 0.00 0 0.00 0.00 0 0.00 0.50 7484 3.65 1 0 27 4 68 152 78 0.00 0 0.00 0.00 0 0.00 0.50 7471 3.65 0 0 29 6 64 287 78 0.00 0 0.00 0.00 0 0.00 0.50 7476 3.65 0 0 28 4 68 2) dd if=/dev/ad8 of=/dev/null bs=128k (/dev/null is on ar0) thor# iostat -c 25 tty ad4 ad6 ad8 cpu tin tout KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy in id 75 35 2.57 0 0.00 2.61 0 0.00 13.20 424 5.46 3 0 2 0 95 1 231 0.00 0 0.00 0.00 0 0.00 64.00 913 57.09 1 0 3 0 95 415 77 0.00 0 0.00 0.00 0 0.00 64.00 912 57.01 0 0 2 0 98 64 77 0.00 0 0.00 0.00 0 0.00 64.00 914 57.14 0 0 2 0 98 16 77 0.00 0 0.00 0.00 0 0.00 64.00 914 57.14 0 0 4 0 96 0 77 0.00 0 0.00 0.00 0 0.00 64.00 912 57.01 0 0 2 0 97 0 77 0.00 0 0.00 0.00 0 0.00 64.00 914 57.14 0 0 3 0 97 96 77 0.00 0 0.00 0.00 0 0.00 64.00 906 56.64 0 0 2 2 96 8 77 0.00 0 0.00 0.00 0 0.00 64.00 905 56.57 0 0 3 0 96 48 77 0.00 0 0.00 0.00 0 0.00 64.00 914 57.14 0 0 2 0 98 431 77 0.00 0 0.00 0.00 0 0.00 64.00 913 57.07 0 0 4 0 96 144 77 0.00 0 0.00 0.00 0 0.00 64.00 914 57.14 0 0 2 0 98 104 77 0.00 0 0.00 0.00 0 0.00 64.00 913 57.07 0 0 2 1 96 16 77 0.00 0 0.00 0.00 0 0.00 64.00 913 57.07 0 0 2 1 97 0 77 0.00 0 0.00 0.00 0 0.00 64.00 914 57.14 0 0 2 0 97 0 77 0.00 0 0.00 0.00 0 0.00 64.00 913 57.07 0 0 3 1 96 1 77 0.00 0 0.00 0.00 0 0.00 64.00 913 57.07 0 0 2 0 98 15 77 0.00 0 0.00 0.00 0 0.00 64.00 914 57.14 0 0 2 2 96 As we can see in this case, bandwith hits the maximum what could be delivered by the SAMSUNG SP2004C drive. 3) dd if=/dev/ad4 of=/dev/null thor# iostat -c 25 tty ad4 ad6 ad8 cpu tin tout KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy in id 79 34 32.72 31 1.00 2.61 0 0.00 15.83 425 6.57 3 0 2 0 95 1 230 0.50 14370 7.02 0.00 0 0.00 0.00 0 0.00 1 0 42 8 50 0 79 0.50 14371 7.02 0.00 0 0.00 0.00 0 0.00 2 0 44 8 46 0 79 0.50 14277 6.97 0.00 0 0.00 0.00 0 0.00 2 0 45 8 45 0 79 0.50 14263 6.96 0.00 0 0.00 0.00 0 0.00 1 0 36 8 55 0 79 0.50 14387 7.03 0.00 0 0.00 0.00 0 0.00 2 0 43 10 44 0 79 0.50 14383 7.02 0.00 0 0.00 0.00 0 0.00 1 0 38 9 52 0 79 0.50 14395 7.03 0.00 0 0.00 0.00 0 0.00 0 0 36 11 53 431 79 0.50 14358 7.01 0.00 0 0.00 0.00 0 0.00 2 0 48 8 42 399 79 0.50 14330 7.00 0.00 0 0.00 0.00 0 0.00 1 0 44 6 50 0 79 0.50 14380 7.02 0.00 0 0.00 0.00 0 0.00 1 0 40 6 52 33 79 0.50 14363 7.01 0.00 0 0.00 0.00 0 0.00 0 0 44 6 50 17 79 0.50 14378 7.02 0.00 0 0.00 0.00 0 0.00 0 0 39 12 49 4) dd if=/dev/ad4 of=/dev/null bs=128k thor# iostat -c 25 tty ad4 ad6 ad8 cpu tin tout KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy in id 78 35 63.44 1 0.05 2.61 0 0.00 15.83 432 6.67 3 0 2 0 95 1 231 64.00 1074 67.14 0.00 0 0.00 0.00 0 0.00 0 0 3 1 95 0 78 64.00 1076 67.24 0.00 0 0.00 0.00 0 0.00 0 0 1 1 98 0 78 64.00 1076 67.24 0.00 0 0.00 0.00 0 0.00 0 0 2 0 97 0 78 64.00 1076 67.24 0.00 0 0.00 0.00 0 0.00 0 0 4 0 95 0 78 64.00 1076 67.24 0.00 0 0.00 0.00 0 0.00 0 0 0 0 99 0 78 64.00 1075 67.18 0.00 0 0.00 0.00 0 0.00 0 0 4 0 95 0 78 64.00 1076 67.24 0.00 0 0.00 0.00 0 0.00 0 0 6 1 93 0 78 64.00 1076 67.24 0.00 0 0.00 0.00 0 0.00 0 0 0 1 99 0 78 64.00 1075 67.18 0.00 0 0.00 0.00 0 0.00 1 0 5 0 93 0 78 64.00 1068 66.74 0.00 0 0.00 0.00 0 0.00 0 0 2 0 98 0 78 64.00 1074 67.12 0.00 0 0.00 0.00 0 0.00 0 0 4 1 94 0 78 64.00 1076 67.24 0.00 0 0.00 0.00 0 0.00 0 0 4 0 96 0 78 64.00 1076 67.24 0.00 0 0.00 0.00 0 0.00 0 0 1 0 99 407 78 64.00 1076 67.24 0.00 0 0.00 0.00 0 0.00 0 0 4 0 96 260 78 64.00 1077 67.30 0.00 0 0.00 0.00 0 0.00 0 0 2 2 96 ==> for /dev/ad6 it is exactly the same! 5) dd if=/dev/ar0 of=/dev/null bs=128k 4 users Load 0.06 0.11 0.06 Mar 4 16:50 Mem:KB REAL VIRTUAL VN PAGER SWAP PAGER Tot Share Tot Share Free in out in out Act 153036 46624 424148 57400 1731088 count All 196176 47268 4685068 69708 pages Proc: Interrupts r p d s w Csw Trp Sys Int Sof Flt cow 5176 total 2 57 10k 27 2307 2166 2330 zfod 5 atkbd0 1 ozfod fdc0 irq6 5.0%Sys 1.5%Intr 0.0%User 0.0%Nice 93.5%Idle %ozfod ata0 irq14 | | | | | | | | | | | daefr atapci2 20 === prcfr 1 nfe0 ohci0 5 dtbuf totfr ehci0 22 Namei Name-cache Dir-cache 100000 desvn react 2160 atapci1 23 Calls hits % hits % 2523 numvn pdwak 3010 cpu0: time 49 49 100 187 frevn pdpgs intrn Disks ad4 ad6 ad8 ar0 cd0 pass0 119912 wire KB/t 0.00 0.00 0.00 64.00 0.00 0.00 109532 act tps 0 0 0 2152 0 0 51188 inact MB/s 0.00 0.00 0.00 135 0.00 0.00 4660 cache <<<<=========!!! %busy 0 0 0 97 0 0 1726428 free 93360 buf Sorry for the messed up cut-and-paste tty output, but as you can see, pretty fast 135 MB/s with 'dd'. This is the systat-output copying an 1G in size QEMU disk image via 'cp' from /dev/ad8 (/dev/ufs/BACKUP) to my home, located on /dev/ar0s3d: 4 users Load 0.11 0.08 0.06 Mar 4 16:57 Mem:KB REAL VIRTUAL VN PAGER SWAP PAGER Tot Share Tot Share Free in out in out Act 130784 28360 413124 57056 90404 count 6 All 173300 29004 4674044 69364 pages 22 Proc: Interrupts r p d s w Csw Trp Sys Int Sof Flt 37 cow 4549 total 2 55 6637 607 2004 1539 1244 110 50 zfod atkbd0 1 50 ozfod fdc0 irq6 14.0%Sys 3.0%Intr 0.0%User 0.0%Nice 83.0%Idle 100%ozfod ata0 irq14 | | | | | | | | | | | daefr 495 atapci2 20 =======++ 57 prcfr 2 nfe0 ohci0 42 dtbuf 16105 totfr ehci0 22 Namei Name-cache Dir-cache 100000 desvn react 1042 atapci1 23 Calls hits % hits % 2626 numvn 1 pdwak 3010 cpu0: time 64 64 100 1804 frevn 12961 pdpgs intrn Disks ad4 ad6 ad8 ar0 cd0 pass0 264168 wire KB/t 0.00 0.00 63.74 58.47 0.00 0.00 134896 act tps 0 0 492 551 0 0 1514576 inact MB/s 0.00 0.00 30.65 31.48 0.00 0.00 87564 cache %busy 0 0 70 53 0 0 2840 free 218608 buf The transfer rate is jumping between 28 MB/s and sometimes it reaches 38 MB/s, but most time it is around 32/33 MB/s. The next tests were taken on the mentioned FreeBSD 6.2/i386 box at my lab. FreeBSD iota.fu-berlin.de 6.2-STABLE FreeBSD 6.2-STABLE #5: Fri Mar 2 10:09:06 CET 2007 root@iota.fu-berlin.de:/usr/obj/usr/src/sys/IOTA i386 Portions of interest taken from dmesg: Copyright (c) 1992-2007 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 6.2-STABLE #5: Fri Mar 2 10:09:06 CET 2007 root@telesto.geoinf.fu-berlin.de:/usr/obj/usr/src/sys/TELESTO Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Pentium(R) 4 CPU 3.00GHz (2998.57-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0xf29 Stepping = 9 Features=0xbfebfbff Features2=0x4400> Logical CPUs per core: 2 real memory = 2146631680 (2047 MB) avail memory = 2095501312 (1998 MB) ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 ioapic0 irqs 0-23 on motherboard kbd1 at kbdmux0 netsmb_dev: loaded acpi0: on motherboard acpi0: Power Button (fixed) Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0 cpu0: on acpi0 cpu1: on acpi0 [...] atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xfc00-0xfc0f at device 31.1 on pci0 ata0: on atapci0 ata1: on atapci0 atapci1: port 0xefe0-0xefe7,0xefac-0xefaf,0xefa0-0xefa7,0xefa8-0xefab,0xef90-0xef9f irq 18 at device 31.2 on pci0 ata2: on atapci1 ata3: on atapci1 [...] acd0: CDRW at ata1-master UDMA33 ad4: 238475MB at ata2-master SATA150 ad6: 381554MB at ata3-master SATA150 [...] atacontrol cap ad4 Protocol Serial ATA II device model Hitachi HDT725025VLA380 serial number VFA100R1CD9VBA firmware revision V5DOA52A cylinders 16383 heads 16 sectors/track 63 lba supported 268435455 sectors lba48 supported 488397168 sectors dma supported overlap not supported Feature Support Enable Value Vendor write cache yes yes read ahead yes yes Native Command Queuing (NCQ) yes - 31/0x1F Tagged Command Queuing (TCQ) no no 31/0x1F SMART yes yes microcode download yes yes security yes no power management yes yes advanced power management yes no 0/0x00 automatic acoustic management yes no 254/0xFE 128/0x80 atacontrol cap ad6 Protocol Serial ATA II device model SAMSUNG HD401LJ serial number S0HVJ1CLB07119 firmware revision ZZ100-15 cylinders 16383 heads 16 sectors/track 63 lba supported 268435455 sectors lba48 supported 781422768 sectors dma supported overlap not supported Feature Support Enable Value Vendor write cache yes yes read ahead yes yes Native Command Queuing (NCQ) yes - 31/0x1F Tagged Command Queuing (TCQ) no no 31/0x1F SMART yes yes microcode download yes yes security yes no power management yes yes advanced power management no no 0/0x00 automatic acoustic management yes no 0/0x00 254/0xFE 'dd' test, shotened, because they reflect the same situation as shown above for the AMD64 box: dd if=/dev/ad4 of=/dev/null bs=128k hartmann: iostat -c 10 tty ad4 ad6 cd0 cpu tin tout KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy in id 0 2 31.12 2 0.06 124.95 1 0.13 0.00 0 0.00 2 0 0 0 98 0 232 128.00 489 61.08 0.00 0 0.00 0.00 0 0.00 0 0 3 1 96 0 78 128.00 505 63.19 0.00 0 0.00 0.00 0 0.00 0 0 5 0 95 0 78 128.00 505 63.19 0.00 0 0.00 0.00 0 0.00 0 0 1 4 95 0 78 128.00 505 63.19 0.00 0 0.00 0.00 0 0.00 0 0 2 1 97 0 78 128.00 499 62.44 0.00 0 0.00 0.00 0 0.00 0 0 2 1 97 0 78 128.00 502 62.81 0.00 0 0.00 0.00 0 0.00 0 0 0 3 97 0 78 128.00 505 63.19 0.00 0 0.00 0.00 0 0.00 0 0 0 3 97 0 78 128.00 505 63.18 0.00 0 0.00 0.00 0 0.00 0 0 2 1 97 0 78 127.78 501 62.45 0.00 0 0.00 0.00 0 0.00 0 0 3 1 97 and dd if=/dev/ad6 of=/dev/null bs=128 hartmann: iostat -c 10 tty ad4 ad6 cd0 cpu tin tout KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy in id 0 2 37.21 2 0.08 124.97 1 0.13 0.00 0 0.00 2 0 0 0 98 0 232 0.00 0 0.00 128.00 551 68.94 0.00 0 0.00 0 0 2 0 98 0 78 0.00 0 0.00 128.00 549 68.68 0.00 0 0.00 0 0 1 1 98 0 78 0.00 0 0.00 128.00 563 70.43 0.00 0 0.00 0 0 3 1 96 0 78 0.00 0 0.00 128.00 559 69.93 0.00 0 0.00 0 0 2 2 96 0 78 0.00 0 0.00 128.00 558 69.81 0.00 0 0.00 0 0 2 2 97 0 78 0.00 0 0.00 128.00 574 71.80 0.00 0 0.00 0 0 5 1 94 0 78 0.00 0 0.00 128.00 547 68.43 0.00 0 0.00 0 0 3 1 96 0 78 0.00 0 0.00 128.00 564 70.55 0.00 0 0.00 0 0 2 2 97 0 78 0.00 0 0.00 128.00 574 71.80 0.00 0 0.00 0 0 3 2 95 And this is the result copying copying a 6.1 G in size qemu drive image from ad4 to ad6: cp qemu/c.img /mnt/ext/s1/qemu/c.img.test hartmann: iostat -c 10 tty ad4 ad6 cd0 cpu tin tout KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy in id 0 2 37.49 2 0.08 126.30 2 0.24 0.00 0 0.00 2 0 0 0 98 0 232 127.66 329 40.99 127.66 329 40.99 0.00 0 0.00 0 0 12 2 86 0 79 126.93 322 39.87 127.30 320 39.74 0.00 0 0.00 0 0 14 2 83 0 79 127.66 329 40.97 127.66 329 40.97 0.00 0 0.00 0 0 12 1 87 0 79 127.66 326 40.60 127.66 326 40.60 0.00 0 0.00 0 0 13 2 85 0 79 127.66 327 40.71 127.66 327 40.71 0.00 0 0.00 0 0 13 3 84 0 79 127.30 320 39.75 126.53 322 39.76 0.00 0 0.00 0 0 11 1 88 0 79 124.90 289 35.21 109.45 337 35.99 0.00 0 0.00 0 0 12 3 85 0 79 127.66 327 40.73 127.66 326 40.60 0.00 0 0.00 0 0 15 1 84 0 79 127.64 312 38.85 127.64 312 38.85 0.00 0 0.00 0 0 11 1 88 It is very interesting to see that FreeBSD 6.2-STABLE/i386 (SMP by HTT) shows up the wrong KB/t size of 128 KB, while FreeBSD 7.0-CURRENT/AMD64 (UP) shows the correct KB/t size of 64 KB/t as mentioned before in this thread. I'm a little bit suprised by the 'good' results of 'cp', because when I did these tests with 'cp' and copying the big files (5 and 6 G qemu drive images) systat hits consequently the 33 MB/s boundary - but I must confess that I had loaded the "aio.ko" kernel module and ran before qemu, kldunloaded kqemu.ko but left aio.ko kldloaded. I will do the tests tomorrow when I'm back at my lab. Nevertheless, on the RAID 0 box I would expect a much better result than on the single-drive boxes. 'dd' seems to do a well job, but it runs as 'root' and I guess a lot of us doing a lot of file copying via 'cp', so there is still a big gap left I can not explain. Sorry for the huge output ... Regards, Oliver