From owner-freebsd-scsi@FreeBSD.ORG Mon May 26 11:06:55 2008 Return-Path: Delivered-To: freebsd-scsi@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 763511065672 for ; Mon, 26 May 2008 11:06:55 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 61F258FC1C for ; Mon, 26 May 2008 11:06:55 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.2/8.14.2) with ESMTP id m4QB6tww065034 for ; Mon, 26 May 2008 11:06:55 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.2/8.14.1/Submit) id m4QB6sc6065030 for freebsd-scsi@FreeBSD.org; Mon, 26 May 2008 11:06:54 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 26 May 2008 11:06:54 GMT Message-Id: <200805261106.m4QB6sc6065030@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-scsi@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-scsi@FreeBSD.org X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 May 2008 11:06:55 -0000 Current FreeBSD problem reports Critical problems Serious problems S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/39388 scsi ncr/sym drivers fail with 53c810 and more than 256MB m o kern/40895 scsi wierd kernel / device driver bug o kern/52638 scsi [panic] SCSI U320 on SMP server won't run faster than s kern/57398 scsi [mly] Current fails to install on mly(4) based RAID di o kern/60598 scsi wire down of scsi devices conflicts with config o kern/60641 scsi [sym] Sporadic SCSI bus resets with 53C810 under load s kern/61165 scsi [panic] kernel page fault after calling cam_send_ccb o kern/74627 scsi [ahc] [hang] Adaptec 2940U2W Can't boot 5.3 o kern/90282 scsi [sym] SCSI bus resets cause loss of ch device o kern/92798 scsi [ahc] SCSI problem with timeouts o kern/94838 scsi Kernel panic while mounting SD card with lock switch o o kern/99954 scsi [ahc] reading from DVD failes on 6.x [regression] o kern/110847 scsi [ahd] Tyan U320 onboard problem with more than 3 disks o kern/120247 scsi [mpt] FreeBSD 6.3 and LSI Logic 1030 = only 3.300MB/s 14 problems total. Non-critical problems S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/35234 scsi World access to /dev/pass? (for scanner) requires acce o kern/38828 scsi [dpt] [request] DPT PM2012B/90 doesn't work o kern/44587 scsi dev/dpt/dpt.h is missing defines required for DPT_HAND o kern/76178 scsi [ahd] Problem with ahd and large SCSI Raid system o kern/114597 scsi [sym] System hangs at SCSI bus reset with dual HBAs o kern/119668 scsi [cam] [patch] certain errors are too verbose comparing o kern/120487 scsi [sg] scsi_sg incompatible with scanners o sparc/121676 scsi [iscsi] iscontrol do not connect iscsi-target on sparc o kern/123666 scsi [aac] aac(4) will not work with Adaptec SAS RAID 3805 o kern/123674 scsi [ahc] ahc driver dumping 10 problems total. From owner-freebsd-scsi@FreeBSD.ORG Mon May 26 13:09:57 2008 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 853301065672 for ; Mon, 26 May 2008 13:09:57 +0000 (UTC) (envelope-from alexander.gsander@gmail.com) Received: from ti-out-0910.google.com (ti-out-0910.google.com [209.85.142.189]) by mx1.freebsd.org (Postfix) with ESMTP id CCD7D8FC16 for ; Mon, 26 May 2008 13:09:56 +0000 (UTC) (envelope-from alexander.gsander@gmail.com) Received: by ti-out-0910.google.com with SMTP id d27so1191126tid.3 for ; Mon, 26 May 2008 06:09:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:mime-version:content-type; bh=FzgbSv+hPqjDHAyn5MxyYVCjRCQPOExxbLVQ6A0VJhs=; b=K+sk2Hefz+neOdDL3okY4kMlUAvJn8HaKCe/ErvrBv9BDLEnJasfBODiZHs25Q06mP2JuYMW+fkTH0YBeOKCrOa8ksCp8X6sZdykEIrJepzW3Cx3yV/qwS+Ns6BMhjNaSwugL4uI8KW4WovA+TIllFqxmAj+/G6HWn4fvat5XcY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:mime-version:content-type; b=tnP4KlrGjTchX2j+JcYEBINkiEbA7pX6eP82q2ovF+7HgKRGpPCMmko8OKxf+7aDpjuuV5h+Gd2GEmyz28Rip6HzMx9NWzaePaPyHjPowOQhHoC704klb9EZkBwDg012tm3+YgAcJp5VcIRuXVr4yyV7iSj4L0zfnkiZG9anGX8= Received: by 10.110.84.3 with SMTP id h3mr15807tib.43.1211805716854; Mon, 26 May 2008 05:41:56 -0700 (PDT) Received: by 10.110.5.6 with HTTP; Mon, 26 May 2008 05:41:56 -0700 (PDT) Message-ID: <5c29cc10805260541r1f90f516q8ac7d45d2bb15702@mail.gmail.com> Date: Mon, 26 May 2008 23:41:56 +1100 From: "Alexander Goncharov" To: freebsd-scsi@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=WINDOWS-1252 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Strange behavior of SCSI RAID 10 (FreeBSD, LSILogic MegaRAID) X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 May 2008 13:09:57 -0000 Hello world, I have faced with following issue on my dedicated server: 8x Opteron 885, 32gm RAM, 8x36 GM 15k rpm SCSI with RAID 10 FreeBSD 7.0-generic, 64 bit version IO performance and behavior is very strange: 1) No other process are running: Memory stat: Mem: 8796K Active, 9372K Inact, 80M Wired, 36K Cache, 12M Buf, 31G Free Copy 3GB file first time dd if=3D/home/3gb_file of=3D/home/3gb_file2 6291456+0 records in 6291456+0 records out 3221225472 bytes transferred in 138.842926 secs (23200501 bytes/sec) 20MBS is very poor=85 Memory stat now: Mem: 8940K Active, 5951M Inact, 287M Wired, 36K Cache, 214M Buf, 25G Free 2) Copy the same file again: dd if=3D/home/3gb_file of=3D/home/3gb_file2 6291456+0 records in 6291456+0 records out 3221225472 bytes transferred in 30.433515 secs (105844674 bytes/sec) 100MBs =96 much better Mem: 9048K Active, 5951M Inact, 287M Wired, 32K Cache, 214M Buf, 25G Free Next attempts with this file show 100MBs spped 3)Copy other file first time dd if=3D/home/test2 of=3D/home/test2_2 6144000+0 records in 6144000+0 records out 3145728000 bytes transferred in 141.870921 secs (22173170 bytes/sec) 20MBs again Copy the same file again: dd if=3D/home/test2 of=3D/home/test2_2 6144000+0 records in 6144000+0 records out 3145728000 bytes transferred in 29.560267 secs (106417441 bytes/sec) 100MBs =96 much better. Memory stat: Mem: 8964K Active, 12G Inact, 287M Wired, 28K Cache, 214M Buf, 19G Free Copy first file again: dd if=3D/home/3gb_file of=3D/home/3gb_file2 6291456+0 records in 6291456+0 records out 3221225472 bytes transferred in 34.310753 secs (93883847 bytes/sec) good speed .... So, first copying of any file is extremely slow, second and all next attempts are much better. Please help me with this issue. Dmesg some info: ioapic0 irqs 0-23 on motherboard kbd1 at kbdmux0 ath_hal: 0.9.20.3 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413, RF5413) hptrr: HPT RocketRAID controller driver v1.1 (May 22 2008 00:49:14) acpi0: on motherboard acpi0: [ITHREAD] acpi0: Power Button (fixed) Timecounter "ACPI-safe" frequency 3579545 Hz quality 850 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x8008-0x800b on acpi0 amr0: mem 0xdc400000-0xdc40ffff,0xdc100000-0xdc13ffff irq 16 at device 14.0 on pci140 amr0: Using 64-bit DMA amr0: [ITHREAD] amr0: delete logical drives supported by controller amr0: Firmware 514L, BIOS H430, 128MB RAM hptrr: no controller detected. acd0: DVDROM at ata1-slave UDMA33 amr0: delete logical drives supported by controller amrd0: on amr0 amrd0: 137328MB (281247744 sectors) RAID 1 (optimal) Best regards, Alexander. From owner-freebsd-scsi@FreeBSD.ORG Mon May 26 14:10:07 2008 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 23EB91065671 for ; Mon, 26 May 2008 14:10:07 +0000 (UTC) (envelope-from gpalmer@freebsd.org) Received: from noop.in-addr.com (in-addr.broker.freenet6.net [IPv6:2001:5c0:8fff:fffe::214d]) by mx1.freebsd.org (Postfix) with ESMTP id E72798FC20 for ; Mon, 26 May 2008 14:10:06 +0000 (UTC) (envelope-from gpalmer@freebsd.org) Received: from gjp by noop.in-addr.com with local (Exim 4.54 (FreeBSD)) id 1K0dOn-000MWz-CM; Mon, 26 May 2008 10:10:05 -0400 Date: Mon, 26 May 2008 10:10:05 -0400 From: Gary Palmer To: Alexander Goncharov Message-ID: <20080526141005.GC1142@in-addr.com> References: <5c29cc10805260541r1f90f516q8ac7d45d2bb15702@mail.gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5c29cc10805260541r1f90f516q8ac7d45d2bb15702@mail.gmail.com> Cc: freebsd-scsi@freebsd.org Subject: Re: Strange behavior of SCSI RAID 10 (FreeBSD, LSILogic MegaRAID) X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 May 2008 14:10:07 -0000 On Mon, May 26, 2008 at 11:41:56PM +1100, Alexander Goncharov wrote: > Hello world, I have faced with following issue on my dedicated server: > > 8x Opteron 885, 32gm RAM, 8x36 GM 15k rpm SCSI with RAID 10 > > FreeBSD 7.0-generic, 64 bit version > > > > IO performance and behavior is very strange: > > 1) No other process are running: > > Memory stat: > > Mem: 8796K Active, 9372K Inact, 80M Wired, 36K Cache, 12M Buf, 31G Free > > Copy 3GB file first time > > dd if=/home/3gb_file of=/home/3gb_file2 > 6291456+0 records in > 6291456+0 records out > 3221225472 bytes transferred in 138.842926 secs (23200501 bytes/sec) > > 20MBS is very poor? > > Memory stat now: > > Mem: 8940K Active, 5951M Inact, 287M Wired, 36K Cache, 214M Buf, 25G Free > > 2) Copy the same file again: > > dd if=/home/3gb_file of=/home/3gb_file2 > 6291456+0 records in > 6291456+0 records out > 3221225472 bytes transferred in 30.433515 secs (105844674 bytes/sec) > > 100MBs ? much better > The "Inact" (Inactive) went up - the 3GiB file is now cached in memory. So the second (and subsequent) runs are going from cached memory so your 100MiB/sec transfer is actually just testing write speed, not read/write speed. This is the same for your other tests too "Inact" is memory that has been used and is being kept around incase it is used again, in other words its caching file data in the "Inact" region in top. It'll be reused if something else needs the memory, but until then it sticks around. Remember - you are copying the file from and to the same filesystem - this is always going to appear slow relative to pure read or pure write tests. A pure write test is effectively what you have when you're getting your 100MiB/sec test result since its just writing out from cache memory. Regards, Gary From owner-freebsd-scsi@FreeBSD.ORG Mon May 26 15:36:45 2008 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 28463106566B for ; Mon, 26 May 2008 15:36:45 +0000 (UTC) (envelope-from alexander.gsander@gmail.com) Received: from ti-out-0910.google.com (ti-out-0910.google.com [209.85.142.190]) by mx1.freebsd.org (Postfix) with ESMTP id 9D0828FC14 for ; Mon, 26 May 2008 15:36:44 +0000 (UTC) (envelope-from alexander.gsander@gmail.com) Received: by ti-out-0910.google.com with SMTP id d27so1243362tid.3 for ; Mon, 26 May 2008 08:36:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:references; bh=Tvg46pRi++TPS9o7NU2PeLn/iFyLfCImE2icsgCMOUw=; b=OtqVzzvCVAyjl3uk5g3E5/lEE6QnFcJAoQUn/F6wzWBbAG1WuuiZX5LFJWvvqxCLxHGpUWW9H4uQ4fZHroncQG79lUriBYNiU+VQ0FLTATzdnJMLRBR6xnFQYsgIGFZIgP5tiaxt9yhSX1IP8Ued/Oy6vbMeCAN9+JcTBLhyfOE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:in-reply-to:mime-version:content-type:references; b=NYmY7dEmINt4HwUh4J+cmgNRjujzRZpdcmzRIRu6CE1Okp+zhDI0Ma1SOMwAkDEGm/Grx0pt3VYjJVNuEPk+En6tf0JSzl5+AxNm7KXySbfIuPV3EE9mC5TfIh3d4WbP73r8Egl8lMEY8DTAUCTvwdQgJJAQi1rsoCuvKQIoQLs= Received: by 10.110.3.15 with SMTP id 15mr40823tic.15.1211816203210; Mon, 26 May 2008 08:36:43 -0700 (PDT) Received: by 10.110.5.6 with HTTP; Mon, 26 May 2008 08:36:43 -0700 (PDT) Message-ID: <5c29cc10805260836rb8ccac0ic722c255e60286f8@mail.gmail.com> Date: Tue, 27 May 2008 02:36:43 +1100 From: "Alexander Goncharov" To: freebsd-scsi@freebsd.org In-Reply-To: <20080526141005.GC1142@in-addr.com> MIME-Version: 1.0 References: <5c29cc10805260541r1f90f516q8ac7d45d2bb15702@mail.gmail.com> <20080526141005.GC1142@in-addr.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: Strange behavior of SCSI RAID 10 (FreeBSD, LSILogic MegaRAID) X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 May 2008 15:36:45 -0000 On Tue, May 27, 2008 at 1:10 AM, Gary Palmer wrote: > On Mon, May 26, 2008 at 11:41:56PM +1100, Alexander Goncharov wrote: > > Hello world, I have faced with following issue on my dedicated server: > > > > 8x Opteron 885, 32gm RAM, 8x36 GM 15k rpm SCSI with RAID 10 > > > > FreeBSD 7.0-generic, 64 bit version > > > > > > > > IO performance and behavior is very strange: > > > > 1) No other process are running: > > > > Memory stat: > > > > Mem: 8796K Active, 9372K Inact, 80M Wired, 36K Cache, 12M Buf, 31G Free > > > > Copy 3GB file first time > > > > dd if=/home/3gb_file of=/home/3gb_file2 > > 6291456+0 records in > > 6291456+0 records out > > 3221225472 bytes transferred in 138.842926 secs (23200501 bytes/sec) > > > > 20MBS is very poor? > > > > Memory stat now: > > > > Mem: 8940K Active, 5951M Inact, 287M Wired, 36K Cache, 214M Buf, 25G Free > > > > 2) Copy the same file again: > > > > dd if=/home/3gb_file of=/home/3gb_file2 > > 6291456+0 records in > > 6291456+0 records out > > 3221225472 bytes transferred in 30.433515 secs (105844674 bytes/sec) > > > > 100MBs ? much better > > > > The "Inact" (Inactive) went up - the 3GiB file is now cached in > memory. So the second (and subsequent) runs are going from cached > memory so your 100MiB/sec transfer is actually just testing write > speed, not read/write speed. This is the same for your other > tests too > > "Inact" is memory that has been used and is being kept around incase > it is used again, in other words its caching file data in the > "Inact" region in top. It'll be reused if something else needs > the memory, but until then it sticks around. > > Remember - you are copying the file from and to the same > filesystem - this is always going to appear slow relative to > pure read or pure write tests. A pure write test is effectively > what you have when you're getting your 100MiB/sec test result since > its just writing out from cache memory. > > Regards, > > Gary > Hi Gary, Big thanks for your quick reply. Read only and write only speed is good ~100MBs. But I am not sure if it really good for hw RAID 10 8x 15k rpm drives. I am worried about read/write speed usecase which is most used at real tasks (data base). I was surprised 20 MBs speed, this value is likely to one drive speed. But I have 8 high speed drives. Something isn't right here. I am waiting for freebsd community help. Thanks in advance. Alexander From owner-freebsd-scsi@FreeBSD.ORG Mon May 26 18:26:23 2008 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8E7D6106567B for ; Mon, 26 May 2008 18:26:23 +0000 (UTC) (envelope-from os@rsu.ru) Received: from mail.r61.net (mail.r61.net [195.208.245.249]) by mx1.freebsd.org (Postfix) with ESMTP id BC3788FC1F for ; Mon, 26 May 2008 18:26:22 +0000 (UTC) (envelope-from os@rsu.ru) Received: from mind.local (os.adsl.r61.net [195.208.243.95]) (authenticated bits=0) by mail.r61.net (8.14.1/8.14.1) with ESMTP id m4QI1pw9005785 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Mon, 26 May 2008 22:01:52 +0400 (MSD) (envelope-from os@rsu.ru) Message-ID: <483AFB0A.6050401@rsu.ru> Date: Mon, 26 May 2008 22:01:46 +0400 From: Oleg Sharoiko User-Agent: Thunderbird 2.0.0.14 (Macintosh/20080421) MIME-Version: 1.0 To: Alexander Goncharov References: <5c29cc10805260541r1f90f516q8ac7d45d2bb15702@mail.gmail.com> <20080526141005.GC1142@in-addr.com> <5c29cc10805260836rb8ccac0ic722c255e60286f8@mail.gmail.com> In-Reply-To: <5c29cc10805260836rb8ccac0ic722c255e60286f8@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-scsi@freebsd.org Subject: Re: Strange behavior of SCSI RAID 10 (FreeBSD, LSILogic MegaRAID) X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 May 2008 18:26:23 -0000 Hi! Alexander Goncharov wrote: > On Tue, May 27, 2008 at 1:10 AM, Gary Palmer wrote: > > >> On Mon, May 26, 2008 at 11:41:56PM +1100, Alexander Goncharov wrote: >> >>> Hello world, I have faced with following issue on my dedicated server: >>> >>> 8x Opteron 885, 32gm RAM, 8x36 GM 15k rpm SCSI with RAID 10 >>> >>> FreeBSD 7.0-generic, 64 bit version >>> >>> >>> >>> IO performance and behavior is very strange: >>> >>> 1) No other process are running: >>> >>> Memory stat: >>> >>> Mem: 8796K Active, 9372K Inact, 80M Wired, 36K Cache, 12M Buf, 31G Free >>> >>> Copy 3GB file first time >>> >>> dd if=/home/3gb_file of=/home/3gb_file2 >>> 6291456+0 records in >>> 6291456+0 records out >>> 3221225472 bytes transferred in 138.842926 secs (23200501 bytes/sec) >>> >>> 20MBS is very poor? >>> >>> Alexander, you're using default block size of 512 bytes. In this case your hardware has to: while (!eof(source_file)) { locate next 512 byte block read it locate writing position write block } So it does 6291456*2 searches. Moreover you're working with files and thus dd has to go throug filesystem layer which involves additional overhead. Please try increasing block size to, for example, 8 mb (add bs=8m to the arguments of dd) >>> Memory stat now: >>> >>> Mem: 8940K Active, 5951M Inact, 287M Wired, 36K Cache, 214M Buf, 25G Free >>> >>> 2) Copy the same file again: >>> >>> dd if=/home/3gb_file of=/home/3gb_file2 >>> 6291456+0 records in >>> 6291456+0 records out >>> 3221225472 bytes transferred in 30.433515 secs (105844674 bytes/sec) >>> >>> 100MBs ? much better >>> >>> >> The "Inact" (Inactive) went up - the 3GiB file is now cached in >> memory. So the second (and subsequent) runs are going from cached >> memory so your 100MiB/sec transfer is actually just testing write >> speed, not read/write speed. This is the same for your other >> tests too >> >> "Inact" is memory that has been used and is being kept around incase >> it is used again, in other words its caching file data in the >> "Inact" region in top. It'll be reused if something else needs >> the memory, but until then it sticks around. >> >> Remember - you are copying the file from and to the same >> filesystem - this is always going to appear slow relative to >> pure read or pure write tests. A pure write test is effectively >> what you have when you're getting your 100MiB/sec test result since >> its just writing out from cache memory. >> >> Regards, >> >> Gary >> >> > > Hi Gary, > > Big thanks for your quick reply. Read only and write only speed is good > ~100MBs. But I am not sure if it really good for hw RAID 10 8x 15k rpm > drives. > I am worried about read/write speed usecase which is most used at real tasks > (data base). I was surprised 20 MBs speed, this value is likely to one drive > speed. But I have 8 high speed drives. Something isn't right here. > > I am waiting for freebsd community help. > > Thanks in advance. > Alexander > _______________________________________________ > freebsd-scsi@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-scsi > To unsubscribe, send any mail to "freebsd-scsi-unsubscribe@freebsd.org" > From owner-freebsd-scsi@FreeBSD.ORG Wed May 28 03:40:05 2008 Return-Path: Delivered-To: freebsd-scsi@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 12EAC1065675 for ; Wed, 28 May 2008 03:40:05 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id F387A8FC0C for ; Wed, 28 May 2008 03:40:04 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.2/8.14.2) with ESMTP id m4S3e4qO091135 for ; Wed, 28 May 2008 03:40:04 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.2/8.14.1/Submit) id m4S3e4m5091134; Wed, 28 May 2008 03:40:04 GMT (envelope-from gnats) Date: Wed, 28 May 2008 03:40:04 GMT Message-Id: <200805280340.m4S3e4m5091134@freefall.freebsd.org> To: freebsd-scsi@FreeBSD.org From: Alexander Titaev Cc: Subject: Re: kern/123666: [aac] aac(4) will not work with Adaptec SAS RAID 3805 controller X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Alexander Titaev List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 May 2008 03:40:05 -0000 The following reply was made to PR kern/123666; it has been noted by GNATS. From: Alexander Titaev To: bug-followup@FreeBSD.org, romain@blogreen.org Cc: Subject: Re: kern/123666: [aac] aac(4) will not work with Adaptec SAS RAID 3805 controller Date: Wed, 28 May 2008 11:52:54 +0900 Hi it find type volume (single drive) only msk-srv# dmesg | grep ^aac aac0: mem 0xd8400000-0xd85fffff irq 18 at device 14.0 on pci11 aac0: Enabling 64-bit address support aac0: New comm. interface enabled aac0: [ITHREAD] aac0: Adaptec 3805, aac driver 2.0.0-1 aacp0: on aac0 aacp1: on aac0 aacp2: on aac0 aacd0: on aac0 aacd0: 953690MB (1953157120 sectors) msk-srv# uname -mr 7.0-RELEASE-p1 amd64 -- regards, Alexander mailto:tit@ispserver.com From owner-freebsd-scsi@FreeBSD.ORG Wed May 28 06:31:35 2008 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 08CDB106568F for ; Wed, 28 May 2008 06:31:35 +0000 (UTC) (envelope-from netslists@gmail.com) Received: from yw-out-2324.google.com (yw-out-2324.google.com [74.125.46.30]) by mx1.freebsd.org (Postfix) with ESMTP id A8FA68FC2B for ; Wed, 28 May 2008 06:31:34 +0000 (UTC) (envelope-from netslists@gmail.com) Received: by yw-out-2324.google.com with SMTP id 9so1711064ywe.13 for ; Tue, 27 May 2008 23:31:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:user-agent:mime-version:to:cc:subject:references:in-reply-to:content-type:content-transfer-encoding; bh=sidvCipuM/IZ9Ssqn62ZIgy7ScSVFMnd0iQDSASr6VU=; b=h4i8YVsP8D3/KXXw9BFsvqrmrvSU4+NKIHyTrubC7ZmSw8+stzMeXyqrvkxYJzTb3q+/6Pv/MZ9/4VXTE2ZemlBc3cO8IIooXLOo1TKvqntKkVs2uf7zGl2T9uMQCm08kuHsyL5LkTdr0S7vcfET186QwpPb0REqb0aFXqVy8hI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:to:cc:subject:references:in-reply-to:content-type:content-transfer-encoding; b=HhugTpnbWLNMJMv7PviKoHxhBR0sqNOSaFHBsL6GIq4Bk6QrP5lQ+ozkzRpXaMQVQkBkjr8KHIKJxuROZbst0wSvkveaC6SfZIhUPEhbYKAlbhZsxmscugPnC4IUfRJ5yMxpI5NXHKrglOGsEtTVSt0ysvyxXzfDtkAp4Nyw7qI= Received: by 10.150.73.7 with SMTP id v7mr4517316yba.123.1211954617039; Tue, 27 May 2008 23:03:37 -0700 (PDT) Received: from ?192.168.12.8? ( [97.101.40.241]) by mx.google.com with ESMTPS id 6sm20538448ywi.1.2008.05.27.23.03.35 (version=SSLv3 cipher=RC4-MD5); Tue, 27 May 2008 23:03:35 -0700 (PDT) Message-ID: <483CF5AE.3090105@gmail.com> Date: Wed, 28 May 2008 02:03:26 -0400 From: Sten Daniel Soersdal User-Agent: Thunderbird 2.0.0.14 (Windows/20080421) MIME-Version: 1.0 To: Alexander Goncharov References: <5c29cc10805260541r1f90f516q8ac7d45d2bb15702@mail.gmail.com> In-Reply-To: <5c29cc10805260541r1f90f516q8ac7d45d2bb15702@mail.gmail.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable Cc: freebsd-scsi@freebsd.org Subject: Re: Strange behavior of SCSI RAID 10 (FreeBSD, LSILogic MegaRAID) X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 May 2008 06:31:35 -0000 Alexander Goncharov wrote: > Hello world, I have faced with following issue on my dedicated server: >=20 > 8x Opteron 885, 32gm RAM, 8x36 GM 15k rpm SCSI with RAID 10 >=20 > FreeBSD 7.0-generic, 64 bit version >=20 >=20 >=20 > IO performance and behavior is very strange: >=20 > 1) No other process are running: >=20 > Memory stat: >=20 > Mem: 8796K Active, 9372K Inact, 80M Wired, 36K Cache, 12M Buf, 31G Free= >=20 > Copy 3GB file first time >=20 > dd if=3D/home/3gb_file of=3D/home/3gb_file2 > 6291456+0 records in > 6291456+0 records out > 3221225472 bytes transferred in 138.842926 secs (23200501 bytes/sec) >=20 > 20MBS is very poor=E2=80=A6 >=20 > Memory stat now: >=20 > Mem: 8940K Active, 5951M Inact, 287M Wired, 36K Cache, 214M Buf, 25G Fr= ee >=20 > 2) Copy the same file again: >=20 > dd if=3D/home/3gb_file of=3D/home/3gb_file2 > 6291456+0 records in > 6291456+0 records out > 3221225472 bytes transferred in 30.433515 secs (105844674 bytes/sec) >=20 > 100MBs =E2=80=93 much better >=20 > Mem: 9048K Active, 5951M Inact, 287M Wired, 32K Cache, 214M Buf, 25G Fr= ee >=20 > Next attempts with this file show 100MBs spped >=20 > 3)Copy other file first time > dd if=3D/home/test2 of=3D/home/test2_2 > 6144000+0 records in > 6144000+0 records out > 3145728000 bytes transferred in 141.870921 secs (22173170 bytes/sec) >=20 > 20MBs again >=20 > Copy the same file again: >=20 > dd if=3D/home/test2 of=3D/home/test2_2 > 6144000+0 records in > 6144000+0 records out > 3145728000 bytes transferred in 29.560267 secs (106417441 bytes/sec) >=20 > 100MBs =E2=80=93 much better. Memory stat: >=20 > Mem: 8964K Active, 12G Inact, 287M Wired, 28K Cache, 214M Buf, 19G Fre= e >=20 > Copy first file again: >=20 > dd if=3D/home/3gb_file of=3D/home/3gb_file2 > 6291456+0 records in > 6291456+0 records out > 3221225472 bytes transferred in 34.310753 secs (93883847 bytes/sec) >=20 > good speed >=20 > .... >=20 > So, first copying of any file is extremely slow, second and all next > attempts are much better. Please help me with this issue. >=20 > Dmesg some info: >=20 >=20 > ioapic0 irqs 0-23 on motherboard > kbd1 at kbdmux0 > ath_hal: 0.9.20.3 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413, RF54= 13) > hptrr: HPT RocketRAID controller driver v1.1 (May 22 2008 00:49:14) > acpi0: on motherboard > acpi0: [ITHREAD] > acpi0: Power Button (fixed) > Timecounter "ACPI-safe" frequency 3579545 Hz quality 850 > acpi_timer0: <24-bit timer at 3.579545MHz> port 0x8008-0x800b on acpi0 >=20 >=20 > amr0: mem > 0xdc400000-0xdc40ffff,0xdc100000-0xdc13ffff irq 16 at device 14.0 on pc= i140 > amr0: Using 64-bit DMA > amr0: [ITHREAD] > amr0: delete logical drives supported by controller > amr0: Firmware 514L, BIOS H430, 128MB R= AM I'm not sure if it's relevant but the LSILogic MegaRAID cards i have are = not able to do true RAID 1+0. Instead it mirrors and then concatenates=20 which will leave you relatively poor read/write performance (no=20 striping). To do RAID 1+0 on my controllers i had to do mirror in BIOS=20 and then stripe the mirror's using gstripe. Also once you do RAID you have to pay great attention to block=20 sizes/widths/etc and alignment of such. Improper alignment will ruin any = performance benefits your RAID levels may provide. Without Battery backed cache your write performance will be poor. Also 'dd' defaults to 512 byte blocks (poor testing). Just my $0.2 --=20 Sten Daniel Soersdal From owner-freebsd-scsi@FreeBSD.ORG Thu May 29 05:30:08 2008 Return-Path: Delivered-To: freebsd-scsi@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D6D501065671 for ; Thu, 29 May 2008 05:30:08 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id DDA138FC17 for ; Thu, 29 May 2008 05:30:08 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.2/8.14.2) with ESMTP id m4T5U8ip051123 for ; Thu, 29 May 2008 05:30:08 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.2/8.14.1/Submit) id m4T5U869051120; Thu, 29 May 2008 05:30:08 GMT (envelope-from gnats) Date: Thu, 29 May 2008 05:30:08 GMT Message-Id: <200805290530.m4T5U869051120@freefall.freebsd.org> To: freebsd-scsi@FreeBSD.org From: Alexander Titaev Cc: Subject: Re: kern/123666: [aac] aac(4) will not work with Adaptec SAS RAID 3805 controller X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Alexander Titaev List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 May 2008 05:30:08 -0000 The following reply was made to PR kern/123666; it has been noted by GNATS. From: Alexander Titaev To: bug-followup@FreeBSD.org, romain@blogreen.org Cc: Subject: Re: kern/123666: [aac] aac(4) will not work with Adaptec SAS RAID 3805 controller Date: Thu, 29 May 2008 14:28:58 +0900 =C7=E4=F0=E0=E2=F1=F2=E2=F3=E9=F2=E5, bug-followup. but in 7.0-STABLE all worked fine msk-srv# uname -mr 7.0-STABLE amd64 msk-srv# dmesg | grep ^aa aac0: mem 0xd8400000-0xd85fffff irq 18 at device 14.0 o= n pci11 aac0: Enabling 64-bit address support aac0: Enable Raw I/O aac0: Enable 64-bit array aac0: New comm. interface enabled aac0: [ITHREAD] aac0: Adaptec 3805, aac driver 2.0.0-1 aacp0: on aac0 aacp1: on aac0 aacp2: on aac0 aacd0: on aac0 aacd0: 953690MB (1953157120 sectors) aacd1: on aac0 aacd1: 5722190MB (11719045120 sectors) --=20 =D1 =F3=E2=E0=E6=E5=ED=E8=E5=EC, Alexander mailto:tit@ispserver.com