From owner-freebsd-questions@FreeBSD.ORG Sat Feb 7 08:00:39 2009 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C50911065670; Sat, 7 Feb 2009 08:00:39 +0000 (UTC) (envelope-from omerfsen@gmail.com) Received: from fk-out-0910.google.com (fk-out-0910.google.com [209.85.128.184]) by mx1.freebsd.org (Postfix) with ESMTP id 231858FC0A; Sat, 7 Feb 2009 08:00:38 +0000 (UTC) (envelope-from omerfsen@gmail.com) Received: by fk-out-0910.google.com with SMTP id f40so1000301fka.11 for ; Sat, 07 Feb 2009 00:00:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type:content-transfer-encoding; bh=IR5RFwETmeD2WEOCCmVRAyif0UpkDaXDQl9GqWleMR4=; b=IG1c15Dw0iKCvyYfAJbUJ3+8W8YxV3gduJN4R5NCdgbxYIYLAyI3kJ963xswxNzgWf 38QYYDNWn2EC9CGWq8DRcFB229bmAsst4ghxxe+jDpy+OIpWPIeipFCdg6N2jkqBLOXZ GifOU7kI/ehPPcLNulmgO3p3fEmsK0RG6QGkw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type :content-transfer-encoding; b=reg7WSTCoNI2HH58Udb0+I1SyiemhB14f5RJyQNdo8p5n4I6otomca1V7BRXS02dJS RlMLlRM43a81Se4s7L26Pcpf+hfQRhMqba4Xgc76Pm7EnE9zDQ4BXYZynYRpeYWoNEW3 mqPwTWboKjuSEs2Tkxv38x404n8gMnjbE/0PU= MIME-Version: 1.0 Received: by 10.181.5.1 with SMTP id h1mr241014bki.56.1233993637966; Sat, 07 Feb 2009 00:00:37 -0800 (PST) Date: Sat, 7 Feb 2009 10:00:37 +0200 Message-ID: <75a268720902070000r7dbe758aq60454092fcab0198@mail.gmail.com> From: Omer Faruk Sen To: freebsd-questions@freebsd.org, freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Subject: FreeBSD 6.3/7.1 and Linux disk performance test X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Feb 2009 08:00:40 -0000 Hi, I have installed a new server to test performance results. BIOS and RAID BIOS is the latest in this server ( Raid controller is a Intel SRCSASBB8I ) # dmesg |grep -i mfi mfi0: port 0x2000-0x20ff mem 0xb8b00000-0xb8b3ffff,0xb8b40000-0xb8b7ffff irq 16 at device 0.0 on pci10 mfi0: Megaraid SAS driver Ver 3.00 mfi0: 1870 (287314652s/0x0020/info) - Shutdown command received from host mfi0: 1871 (boot + 3s/0x0020/info) - Firmware initialization started (PCI ID 0060/1000/1013/8086) mfi0: 1872 (boot + 3s/0x0020/info) - Firmware version 1.20.72-0562 mfi0: 1873 (boot + 3s/0x0020/info) - Board Revision mfi0: 1874 (boot + 15s/0x0002/info) - Inserted: PD 08(e0xff/s8) mfi0: 1875 (boot + 15s/0x0002/info) - Inserted: PD 08(e0xff/s8) Info: enclPd=ffff, scsiType=0, portMap=00, sasAddr=5000c5000bcb90b1,0000000000000000 mfi0: 1876 (boot + 15s/0x0002/info) - Inserted: PD 09(e0xff/s9) mfi0: 1877 (boot + 15s/0x0002/info) - Inserted: PD 09(e0xff/s9) Info: enclPd=ffff, scsiType=0, portMap=01, sasAddr=5000c5000bcb962d,0000000000000000 mfi0: 1878 (boot + 15s/0x0002/info) - Inserted: PD 0a(e0xff/s10) mfi0: 1879 (boot + 15s/0x0002/info) - Inserted: PD 0a(e0xff/s10) Info: enclPd=ffff, scsiType=0, portMap=02, sasAddr=5000c5000bcb8f8d,0000000000000000 mfi0: [ITHREAD] mfi0: 1880 (287314711s/0x0020/info) - Time established as 02/07/09 9:38:31; (52 seconds since power on) mfid0: on mfi0 mfid0: 556928MB (1140588544 sectors) RAID volume '' I have installed RHEL 5.3 x64 on this server and here is simple dd performance test: Last login: Fri Feb 6 12:15:32 2009 from 10.0.0.51 [root@localhost ~]# dd if=/dev/zero of=bigfile bs=8192 count=100000 100000+0 records in 100000+0 records out 819200000 bytes (819 MB) copied, 1.8473 seconds, 443 MB/s [root@localhost ~]# dd if=/dev/zero of=bigfile bs=8192 count=100000 100000+0 records in 100000+0 records out 819200000 bytes (819 MB) copied, 2.05412 seconds, 399 MB/s [root@localhost ~]# dd if=/dev/zero of=bigfile bs=8192 count=100000 100000+0 records in 100000+0 records out 819200000 bytes (819 MB) copied, 1.94096 seconds, 422 MB/s [root@localhost ~]# dd if=/dev/zero of=bigfile bs=8192 count=100000 100000+0 records in 100000+0 records out 819200000 bytes (819 MB) copied, 2.1092 seconds, 388 MB/s [root@localhost ~]# dd if=/dev/zero of=bigfile bs=8192 count=100000 100000+0 records in 100000+0 records out 819200000 bytes (819 MB) copied, 1.84414 seconds, 444 MB/s [root@localhost ~]# dd if=/dev/zero of=bigfile bs=8192 count=100000 100000+0 records in 100000+0 records out 819200000 bytes (819 MB) copied, 2.09829 seconds, 390 MB/s And after that I have tested with FreeBSD 6.3 and 7.1: 6.3: # dd if=/dev/zero of=bigfile bs=8192 count=100000 100000+0 records in 100000+0 records out 819200000 bytes transferred in 3.534685 secs (231760388 bytes/sec) # cd /var/tmp/ # dd if=/dev/zero of=bigfile bs=8192 count=100000 100000+0 records in 100000+0 records out 819200000 bytes transferred in 3.051668 secs (268443342 bytes/sec) # cd / # cd /var/tmp/ # dd if=/dev/zero of=bigfile bs=8192 count=100000 100000+0 records in 100000+0 records out 819200000 bytes transferred in 3.001266 secs (272951481 bytes/sec) # cd /home/ cd: can't cd to /home/ # cd /usr/ # dd if=/dev/zero of=bigfile bs=8192 count=100000 100000+0 records in 100000+0 records out 819200000 bytes transferred in 3.678156 secs (222720290 bytes/sec) # cd /boot # dd if=/dev/zero of=bigfile bs=8192 count=100000 cd /usr/100000+0 records in 100000+0 records out 819200000 bytes transferred in 2.985471 secs (274395563 bytes/sec) # local # pwd /usr/local # dd if=/dev/zero of=bigfile bs=8192 count=100000 100000+0 records in 100000+0 records out 819200000 bytes transferred in 3.501986 secs (233924406 bytes/sec) 7.1: Filesystem Size Used Avail Capacity Mounted on /dev/mfid0s1a 9.7G 2.6G 6.3G 29% / devfs 1.0K 1.0K 0B 100% /dev /dev/mfid0s1d 478G 4.0K 440G 0% /opt # dd if=/dev/zero of=bigfile bs=8192 count=100000 100000+0 records in 100000+0 records out 819200000 bytes transferred in 3.441476 secs (238037393 bytes/sec) # cd / # dd if=/dev/zero of=bigfile bs=8192 count=100000 100000+0 records in 100000+0 records out 819200000 bytes transferred in 3.132512 secs (261515371 bytes/sec) # cd /usr/local/ # dd if=/dev/zero of=bigfile bs=8192 count=100000 100000+0 records in 100000+0 records out 819200000 bytes transferred in 3.296514 secs (248504951 bytes/sec) # cd /usr # dd if=/dev/zero of=bigfile bs=8192 count=100000 100000+0 records in 100000+0 records out 819200000 bytes transferred in 3.069655 secs (266870386 bytes/sec) as you can see there is a big difference in just simple dd test. Is there additional steps that I can follow to increase performance? By the way If I use write-thru cache on this raid card FreeBSD 6.3 and FreeBSD 7.1 only gives 13MB/s~ which is very very bad. Linux gives a slight decrease on dd test with write-thru cache but freebsd goes from 230-270 MB/s to only 13 MB/s Regards.