From owner-freebsd-scsi@FreeBSD.ORG Mon May 26 13:09:57 2008 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 853301065672 for ; Mon, 26 May 2008 13:09:57 +0000 (UTC) (envelope-from alexander.gsander@gmail.com) Received: from ti-out-0910.google.com (ti-out-0910.google.com [209.85.142.189]) by mx1.freebsd.org (Postfix) with ESMTP id CCD7D8FC16 for ; Mon, 26 May 2008 13:09:56 +0000 (UTC) (envelope-from alexander.gsander@gmail.com) Received: by ti-out-0910.google.com with SMTP id d27so1191126tid.3 for ; Mon, 26 May 2008 06:09:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:mime-version:content-type; bh=FzgbSv+hPqjDHAyn5MxyYVCjRCQPOExxbLVQ6A0VJhs=; b=K+sk2Hefz+neOdDL3okY4kMlUAvJn8HaKCe/ErvrBv9BDLEnJasfBODiZHs25Q06mP2JuYMW+fkTH0YBeOKCrOa8ksCp8X6sZdykEIrJepzW3Cx3yV/qwS+Ns6BMhjNaSwugL4uI8KW4WovA+TIllFqxmAj+/G6HWn4fvat5XcY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:mime-version:content-type; b=tnP4KlrGjTchX2j+JcYEBINkiEbA7pX6eP82q2ovF+7HgKRGpPCMmko8OKxf+7aDpjuuV5h+Gd2GEmyz28Rip6HzMx9NWzaePaPyHjPowOQhHoC704klb9EZkBwDg012tm3+YgAcJp5VcIRuXVr4yyV7iSj4L0zfnkiZG9anGX8= Received: by 10.110.84.3 with SMTP id h3mr15807tib.43.1211805716854; Mon, 26 May 2008 05:41:56 -0700 (PDT) Received: by 10.110.5.6 with HTTP; Mon, 26 May 2008 05:41:56 -0700 (PDT) Message-ID: <5c29cc10805260541r1f90f516q8ac7d45d2bb15702@mail.gmail.com> Date: Mon, 26 May 2008 23:41:56 +1100 From: "Alexander Goncharov" To: freebsd-scsi@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=WINDOWS-1252 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Strange behavior of SCSI RAID 10 (FreeBSD, LSILogic MegaRAID) X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 May 2008 13:09:57 -0000 Hello world, I have faced with following issue on my dedicated server: 8x Opteron 885, 32gm RAM, 8x36 GM 15k rpm SCSI with RAID 10 FreeBSD 7.0-generic, 64 bit version IO performance and behavior is very strange: 1) No other process are running: Memory stat: Mem: 8796K Active, 9372K Inact, 80M Wired, 36K Cache, 12M Buf, 31G Free Copy 3GB file first time dd if=3D/home/3gb_file of=3D/home/3gb_file2 6291456+0 records in 6291456+0 records out 3221225472 bytes transferred in 138.842926 secs (23200501 bytes/sec) 20MBS is very poor=85 Memory stat now: Mem: 8940K Active, 5951M Inact, 287M Wired, 36K Cache, 214M Buf, 25G Free 2) Copy the same file again: dd if=3D/home/3gb_file of=3D/home/3gb_file2 6291456+0 records in 6291456+0 records out 3221225472 bytes transferred in 30.433515 secs (105844674 bytes/sec) 100MBs =96 much better Mem: 9048K Active, 5951M Inact, 287M Wired, 32K Cache, 214M Buf, 25G Free Next attempts with this file show 100MBs spped 3)Copy other file first time dd if=3D/home/test2 of=3D/home/test2_2 6144000+0 records in 6144000+0 records out 3145728000 bytes transferred in 141.870921 secs (22173170 bytes/sec) 20MBs again Copy the same file again: dd if=3D/home/test2 of=3D/home/test2_2 6144000+0 records in 6144000+0 records out 3145728000 bytes transferred in 29.560267 secs (106417441 bytes/sec) 100MBs =96 much better. Memory stat: Mem: 8964K Active, 12G Inact, 287M Wired, 28K Cache, 214M Buf, 19G Free Copy first file again: dd if=3D/home/3gb_file of=3D/home/3gb_file2 6291456+0 records in 6291456+0 records out 3221225472 bytes transferred in 34.310753 secs (93883847 bytes/sec) good speed .... So, first copying of any file is extremely slow, second and all next attempts are much better. Please help me with this issue. Dmesg some info: ioapic0 irqs 0-23 on motherboard kbd1 at kbdmux0 ath_hal: 0.9.20.3 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413, RF5413) hptrr: HPT RocketRAID controller driver v1.1 (May 22 2008 00:49:14) acpi0: on motherboard acpi0: [ITHREAD] acpi0: Power Button (fixed) Timecounter "ACPI-safe" frequency 3579545 Hz quality 850 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x8008-0x800b on acpi0 amr0: mem 0xdc400000-0xdc40ffff,0xdc100000-0xdc13ffff irq 16 at device 14.0 on pci140 amr0: Using 64-bit DMA amr0: [ITHREAD] amr0: delete logical drives supported by controller amr0: Firmware 514L, BIOS H430, 128MB RAM hptrr: no controller detected. acd0: DVDROM at ata1-slave UDMA33 amr0: delete logical drives supported by controller amrd0: on amr0 amrd0: 137328MB (281247744 sectors) RAID 1 (optimal) Best regards, Alexander.