From owner-freebsd-hardware@FreeBSD.ORG Sun Nov 9 23:49:36 2008 Return-Path: Delivered-To: freebsd-hardware@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B2EB6106567E for ; Sun, 9 Nov 2008 23:49:36 +0000 (UTC) (envelope-from joe@zircon.seattle.wa.us) Received: from dsl254-019-221.sea1.dsl.speakeasy.net (dsl254-019-221.sea1.dsl.speakeasy.net [216.254.19.221]) by mx1.freebsd.org (Postfix) with ESMTP id 5DB8C8FC0C for ; Sun, 9 Nov 2008 23:49:36 +0000 (UTC) (envelope-from joe@zircon.seattle.wa.us) Received: (qmail 2801 invoked from network); 9 Nov 2008 23:29:08 -0000 Received: from localhost (HELO zircon.zircon.seattle.wa.us) (127.0.0.1) by localhost with ESMTP; 9 Nov 2008 23:29:08 -0000 Message-ID: <49177244.9060802@zircon.seattle.wa.us> Date: Sun, 09 Nov 2008 15:29:08 -0800 From: Joe Kelsey User-Agent: Thunderbird 2.0.0.17 (X11/20081017) MIME-Version: 1.0 To: =?ISO-8859-1?Q?S=F8ren_Schmidt?= References: <20081107071752.GA5842@icarus.home.lan> <77C223A7-C5FC-45DE-BF1A-3BC7982FA582@FreeBSD.ORG> In-Reply-To: <77C223A7-C5FC-45DE-BF1A-3BC7982FA582@FreeBSD.ORG> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit X-Mailman-Approved-At: Mon, 10 Nov 2008 00:05:27 +0000 Cc: Jeremy Chadwick , freebsd-stable@FreeBSD.ORG, votdev@gmx.de, Peter Wemm , freebsd-hardware@FreeBSD.ORG Subject: Re: Western Digital hard disks and ATA timeouts X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Nov 2008 23:49:36 -0000 Søren Schmidt wrote: > On 7Nov, 2008, at 20:12 , Peter Wemm wrote: > >> On Thu, Nov 6, 2008 at 11:17 PM, Jeremy Chadwick >> wrote: >> [..] >>> As stated, FreeBSD's ATA command timeout is hard-set to 5 seconds, and >>> is not adjustable without editing the ATA code yourself and increasing >>> the value. The FreeNAS folks have made patches available to turn the >>> timeout value into a sysctl. >>> >>> Soren and/or others, please increase this timeout value. Five seconds >>> has now been deemed too aggressive a default. And please consider >>> migrating the timeout value into a sysctl. >> >> The 5 second timeout has been a problem for quite a while actually. >> I've had a number of instances where I've had to increase it to 20 or >> 30 seconds when recovering from marginal drives. The longest >> "successful" recovery attempt I've seen was 26 seconds, I believe on a >> Maxtor drive a few years ago. ("successful" == the drive spent 26 >> seconds but eventually successfully read the sector). Even the IBM >> death star drives could take much longer than 5 seconds to do a >> recovery 5 years ago. 5 seconds has never been a good default. >> >> I think the timeout should be increased to at least 30 seconds. My >> windows box has a timeout that goes for several minutes. >> >> If there is concern about FreeBSD appearing to hang, I could imagine >> that a console warning message could be printed after 5 seconds. But >> just say "drive has not yet responded". But give it more time. >> >> In this day and age we're generally not playing games with udma33 vs >> 66, notched cables, poor CRC support etc. SATA seems to have >> eliminated all that. Hmm, it might make sense to increase the timeout >> on SATA connections to 2 or 3 minutes by default. > > Actually I do have a patch around that logs the timeout on the console > after the normal timeout (5secs), then just goes on to wait for double > the timeout and log again etc etc, final timeout was IIRC 60 secs but > could be anything. I have a disk which I am finally getting rid of that produces READ_DMA and WRITE_DMA errors at a pretty high rate. I did enable the extra ATA error reporting and it doesn't seem to indicate any sort of actual errors, just extra long itmeouts. At one time, I did change the system to extend the timeout, but I did not see any real improvement at 30 seconds. I suspect that an even more extended timeout would be necessary to solve the problem. I am removing the disk this week. Does anyone want a disk that produces DMA timeouts at a regular rate? Would it help actually solve this problem? Please let me know if you want such a beast and I will ship it to you. /Joe From owner-freebsd-hardware@FreeBSD.ORG Tue Nov 11 16:02:26 2008 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 635A4106567B for ; Tue, 11 Nov 2008 16:02:26 +0000 (UTC) (envelope-from roberth.sjonoy@gmail.com) Received: from rn-out-0910.google.com (rn-out-0910.google.com [64.233.170.187]) by mx1.freebsd.org (Postfix) with ESMTP id 1C4AF8FC24 for ; Tue, 11 Nov 2008 16:02:25 +0000 (UTC) (envelope-from roberth.sjonoy@gmail.com) Received: by rn-out-0910.google.com with SMTP id j71so2227255rne.12 for ; Tue, 11 Nov 2008 08:02:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:mime-version:content-type:content-transfer-encoding :content-disposition; bh=/KrnvQEKjrlOoVgXRqRBTUu7eg3EA3c9bmg6wUJ+vzI=; b=Ahm9ORKLrvQKgAbiQ8iFawYaRv3G88WGO8pDia0WUiDDnYje2vI1tXT+PprFY92kN7 B9ORKkO3ESeG0o5ZgX51i4IOsv/bkrQw9xt7qkGkim3gvhtVGD1eBJsuLjY/Dz22h/6W vMOfe5JHFtgQxEQNYfOxbAmBJnCO7CUwT+QoA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:mime-version:content-type :content-transfer-encoding:content-disposition; b=bsNMG6lFCxlZJTIslNObN4b+1RjQAic3f+PcZoxYf5Wm9afpAoBreIt4iuGmHHk/Jv x2H8uATLyTzLm0d83rPsC+m+eGg7lkwlXEfV+cIh+cP/RA3m+RLJra7or3U6rtqo79ee wM+iQ8pI73NyR9LaUI69xXkd8lDN8nY4LYcRU= Received: by 10.90.113.17 with SMTP id l17mr7345701agc.40.1226418018540; Tue, 11 Nov 2008 07:40:18 -0800 (PST) Received: by 10.90.114.6 with HTTP; Tue, 11 Nov 2008 07:40:18 -0800 (PST) Message-ID: Date: Tue, 11 Nov 2008 16:40:18 +0100 From: "=?ISO-8859-1?Q?Roberth_Sjon=F8y?=" To: freebsd-hardware@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Subject: cmi 8788 support? X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Nov 2008 16:02:26 -0000 Hello, I see that FreeBSD 7.1-BETA2 supports creatives X-FI soundcards, but I'm wondering where the support for the widely used cmi8788 chipset is? Regards, Roberth Sjon=F8y From owner-freebsd-hardware@FreeBSD.ORG Wed Nov 12 10:29:19 2008 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F40471065670; Wed, 12 Nov 2008 10:29:18 +0000 (UTC) (envelope-from ltning@anduin.net) Received: from mail.anduin.net (mail.anduin.net [213.225.74.249]) by mx1.freebsd.org (Postfix) with ESMTP id B6F768FC0A; Wed, 12 Nov 2008 10:29:18 +0000 (UTC) (envelope-from ltning@anduin.net) Received: from [193.213.86.45] (helo=[192.168.3.193]) by mail.anduin.net with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.69 (FreeBSD)) (envelope-from ) id 1L0CyK-000A8A-9H; Wed, 12 Nov 2008 11:29:16 +0100 Message-Id: From: =?ISO-8859-1?Q?Eirik_=D8verby?= To: Ivan Voras In-Reply-To: <6D397584-2B6B-4F1A-B5C4-2A7B77AE52EB@anduin.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed; delsp=yes Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Apple Message framework v929.2) Date: Wed, 12 Nov 2008 11:29:15 +0100 References: <6D397584-2B6B-4F1A-B5C4-2A7B77AE52EB@anduin.net> X-Mailer: Apple Mail (2.929.2) Sender: ltning@anduin.net Cc: freebsd-hardware@freebsd.org Subject: Re: FreeBSD-AMD64 on Xeon MP - SOLVED X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Nov 2008 10:29:19 -0000 Hi all, the issue below has been solved by a new BIOS for the server in =20 question. I recently received a beta BIOS from Supermicro, after =20 having reported the issue and done a bit of troubleshooting with them. =20= The tip that helped the most was the boot-linux-first trick. Anyone who wants the BIOS may mail me privately; otherwise it'll =20 likely be released by Supermicro in the not-too-distant future. /Eirik On Oct 20, 2008, at 08:22, Eirik =D8verby wrote: > Hi, > >> > Hi all, >> > >> > I try to run FreeBSD-7-AMD64 on a Quad Xeon (Xeon MP 7320) and =20 >> 32GB RAM. >> > The Board is a X7QC3 by supermicro and the installation is done on >> > another system, updated and plugged to this system. So I have a =20 >> drive >> > with 7-STABLE compiled today. >> > >> > The last line I see from dmesg is vga0- then the system freezes. >> > >> > Anyone using a similar configuration or knows what could be =20 >> wrong? I >> > still have some days left to play with it, before this box gets =20 >> shipped >> > to the customer. >> >> This looks very, very similar to what I had once, on similar hardware >> (4x Xeon 7xxx, SuperMicro). I didn't find a solution and didn't =20 >> bother >> since the box isn't intended for FreeBSD. I did find (by accident) a >> curious workaround: I booted Linux (I used Ubuntu 8.04 amd64 LiveCD - >> just to boot it, without installing), then rebooted and booted =20 >> FreeBSD - >> worked every time, but it's obviously not a long-term solution. If =20= >> you >> can also verify that this "solves" the problem, then someone might =20= >> work >> with you to produce a patch. > > I just received four such servers, all intended for FreeBSD.... And =20= > I'm seeing exactly the same problem. I'm going to try booting Linux =20= > and then back into FreeBSD, but it's obviously not a solution. =20 > Anyone who might want to work on this can have a box like this to =20 > work on via remote KVM (including remote boot media capability) any =20= > time. > > I'm going to go poke the supplier and Supermicro for some updated =20 > firmware. > Any progress on your end? > > Some additional info: Safe mode boot gets a bit further, to the =20 > point where it tries to mount/read from /dev/md0, but then hangs hard. > > /Eirik > From owner-freebsd-hardware@FreeBSD.ORG Wed Nov 12 11:23:31 2008 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2203D1065674 for ; Wed, 12 Nov 2008 11:23:31 +0000 (UTC) (envelope-from ivoras@gmail.com) Received: from fg-out-1718.google.com (fg-out-1718.google.com [72.14.220.156]) by mx1.freebsd.org (Postfix) with ESMTP id A546A8FC0C for ; Wed, 12 Nov 2008 11:23:30 +0000 (UTC) (envelope-from ivoras@gmail.com) Received: by fg-out-1718.google.com with SMTP id l26so330125fgb.35 for ; Wed, 12 Nov 2008 03:23:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:sender :to:subject:cc:in-reply-to:mime-version:content-type :content-transfer-encoding:content-disposition:references :x-google-sender-auth; bh=fewkxdxn/d8D4RzlqqgIw7Y1Ep7uhrnIJJdo5kIW8K0=; b=lv9dxFOppsTJtP01jQPzNZ7PKF+PslldzqevPJMAmv/j1bUTDcib0otRnmomChyuFV CQWNAyfBLOH/FP5qoBPh2S6vHqZFD5JROHIBY6Ws14+pzfS5BsZ3H4efcbw/bw8Yrtpo rRZ9taS9gk0/hSIUlH3KxydY61LBKsAFNaQVg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version :content-type:content-transfer-encoding:content-disposition :references:x-google-sender-auth; b=CXa3aPsjNYbcCsqzbEjfJGSOi9oqQ0hwF4h6ll6NtQ49deXAEkJ0+ep+ipHJ3rNjrv S0DtFT0Ovu2EAIZiESl9nXs9JjV/sZPAqo9P1wB1K/rIyY9YgUJ6wN1zdeTMle3h3vUu LgHt13PEvOD9iLNPzAKFUbG5agyum7OnAItWU= Received: by 10.181.48.13 with SMTP id a13mr2808991bkk.43.1226487065129; Wed, 12 Nov 2008 02:51:05 -0800 (PST) Received: by 10.181.215.20 with HTTP; Wed, 12 Nov 2008 02:51:05 -0800 (PST) Message-ID: <9bbcef730811120251w320e9d0cv972424ab02dc444e@mail.gmail.com> Date: Wed, 12 Nov 2008 11:51:05 +0100 From: "Ivan Voras" Sender: ivoras@gmail.com To: "=?UTF-8?Q?Eirik_=C3=98verby?=" In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: base64 Content-Disposition: inline References: <6D397584-2B6B-4F1A-B5C4-2A7B77AE52EB@anduin.net> X-Google-Sender-Auth: 07cfb214380391bf Cc: freebsd-hardware@freebsd.org Subject: Re: FreeBSD-AMD64 on Xeon MP - SOLVED X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Nov 2008 11:23:31 -0000 MjAwOC8xMS8xMiBFaXJpayDDmHZlcmJ5IDxsdG5pbmdAbWFjLmNvbT46Cj4gSGkgYWxsLAo+Cj4g dGhlIGlzc3VlIGJlbG93IGhhcyBiZWVuIHNvbHZlZCBieSBhIG5ldyBCSU9TIGZvciB0aGUgc2Vy dmVyIGluIHF1ZXN0aW9uLiBJCj4gcmVjZW50bHkgcmVjZWl2ZWQgYSBiZXRhIEJJT1MgZnJvbSBT dXBlcm1pY3JvLCBhZnRlciBoYXZpbmcgcmVwb3J0ZWQgdGhlCj4gaXNzdWUgYW5kIGRvbmUgYSBi aXQgb2YgdHJvdWJsZXNob290aW5nIHdpdGggdGhlbS4gVGhlIHRpcCB0aGF0IGhlbHBlZCB0aGUK PiBtb3N0IHdhcyB0aGUgYm9vdC1saW51eC1maXJzdCB0cmljay4KPgo+IEFueW9uZSB3aG8gd2Fu dHMgdGhlIEJJT1MgbWF5IG1haWwgbWUgcHJpdmF0ZWx5OyBvdGhlcndpc2UgaXQnbGwgbGlrZWx5 IGJlCj4gcmVsZWFzZWQgYnkgU3VwZXJtaWNybyBpbiB0aGUgbm90LXRvby1kaXN0YW50IGZ1dHVy ZS4KCkhpLAoKSSdkIGxpa2UgdGhlIHVwZGF0ZSBzaW5jZSBpdCdzIGxpa2VseSBJJ2xsIGdldCBh bm90aGVyIG9mIHRob3NlIG1hY2hpbmVzIHNvb24uCg== From owner-freebsd-hardware@FreeBSD.ORG Thu Nov 13 05:46:34 2008 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 18E781065670 for ; Thu, 13 Nov 2008 05:46:34 +0000 (UTC) (envelope-from danny@dannysplace.net) Received: from mail.dannysplace.net (mail.dannysplace.net [213.133.54.210]) by mx1.freebsd.org (Postfix) with ESMTP id BB4628FC0C for ; Thu, 13 Nov 2008 05:46:33 +0000 (UTC) (envelope-from danny@dannysplace.net) Received: from 203-206-171-212.perm.iinet.net.au ([203.206.171.212] helo=[192.168.10.10]) by mail.dannysplace.net with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.69 (FreeBSD)) (envelope-from ) id 1L0V25-000IUu-PC; Thu, 13 Nov 2008 15:46:32 +1000 Message-ID: <491BBF38.9010908@dannysplace.net> Date: Thu, 13 Nov 2008 15:46:32 +1000 From: Danny Carroll User-Agent: Thunderbird 2.0.0.17 (Windows/20080914) MIME-Version: 1.0 To: fbsd@dannysplace.net References: <490A782F.9060406@dannysplace.net> <20081031033208.GA21220@icarus.home.lan> <490A849C.7030009@dannysplace.net> <20081031043412.GA22289@icarus.home.lan> <490A8FAD.8060009@dannysplace.net> In-Reply-To: <490A8FAD.8060009@dannysplace.net> X-Enigmail-Version: 0.95.7 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Authenticated-User: danny X-Authenticator: plain X-Sender-Verify: SUCCEEDED (sender exists & accepts mail) X-Exim-Version: 4.69 (build at 08-Jul-2008 08:59:40) X-Date: 2008-11-13 15:46:22 X-Connected-IP: 203.206.171.212:3073 X-Message-Linecount: 85 X-Body-Linecount: 70 X-Message-Size: 2967 X-Body-Size: 2246 X-Received-Count: 1 X-Recipient-Count: 4 X-Local-Recipient-Count: 4 X-Local-Recipient-Defer-Count: 0 X-Local-Recipient-Fail-Count: 0 X-SA-Exim-Connect-IP: 203.206.171.212 X-SA-Exim-Rcpt-To: fbsd@dannysplace.net, koitsu@FreeBSD.org, freebsd-fs@freebsd.org, freebsd-hardware@freebsd.org X-SA-Exim-Mail-From: danny@dannysplace.net X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on ferrari.dannysplace.net X-Spam-Level: X-Spam-Status: No, score=0.2 required=8.0 tests=ALL_TRUSTED,AWL,TVD_RCVD_IP autolearn=disabled version=3.2.5 X-SA-Exim-Version: 4.2 X-SA-Exim-Scanned: Yes (on mail.dannysplace.net) Cc: freebsd-fs@freebsd.org, Jeremy Chadwick , freebsd-hardware@freebsd.org Subject: Re: Areca vs. ZFS performance testing. X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Nov 2008 05:46:34 -0000 Danny Carroll wrote: > Jeremy Chadwick wrote: >> I'd like to see the performance difference between these scenarios: >> >> - Memory cache enabled on Areca, write caching enabled on disks >> - Memory cache enabled on Areca, write caching disabled on disks >> - Memory cache disabled on Areca, write caching enabled on disks >> - Memory cache disabled on Areca, write caching disabled on disks >> The initial results for a ICH9 vs Areca in JBod mode can be found here: http://www.dannysplace.net/ZFS-JBODTests.html Summary: 5 Disk ZFS RaidZ array with atime turned off. ICH9 - block reads avg 400MByte/Sec ICH9 - block writes avg 150MByte/Sec ArecaJBOD - block reads avg 300MByte/Sec ArecaJBOD - block writes avg 160MByte/Sec The Areca seems to be in all except char and block writes. Block reads are 75% as fast as the ICH9 and rewrites are about 85% as fast. There seems to be little difference between enabling and disabling the disk cache on the Areca. This leads me to two conclusions: 1. Disabling the write cache does nothing on Seagate drives. 2. IO to the drives is so slow that a write cache is irrelevant. These are just some quick tests that I started with, mainly to compare the areca bus versus the ich9 bus. If someone has any tuning suggestions, then now is the time to make them before I migrate the ICH9 drives to the Areca bus. -D p.s. My OS details are: FreeBSD 7.1-PRERELEASE #3: Tue Nov 4 13:58:49 EST 2008 localhost# cat /etc/sysctl.conf kern.maxvnodes=400000 net.key.preferred_oldsa=0 net.key.blockacq_count=0 kern.ipc.maxsockbuf=400000 net.inet.ip.fastforwarding=1 net.inet.tcp.rfc1323=1 kern.ipc.maxsockbuf=16777216 net.local.stream.sendspace=82320 net.local.stream.recvspace=82320 net.inet.tcp.local_slowstart_flightsize=10 net.inet.tcp.nolocaltimewait=1 net.inet.tcp.delayed_ack=1 net.inet.tcp.delacktime=100 net.inet.tcp.mssdflt=1460 net.inet.tcp.sendspace=78840 net.inet.tcp.recvspace=78840 net.inet.tcp.slowstart_flightsize=54 net.inet.tcp.inflight.enable=1 net.inet.tcp.inflight.min=6144 net.inet.tcp.hostcache.expire=3900 localhost# cat /boot/loader.conf hw.em.rxd=4096 hw.em.txd=4096 vm.kmem_size="1536M" vm.kmem_size_max="1536M" smb_load="YES" smbus_load="YES" ichsmb_load="YES" From owner-freebsd-hardware@FreeBSD.ORG Thu Nov 13 06:35:57 2008 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0C683106567D for ; Thu, 13 Nov 2008 06:35:57 +0000 (UTC) (envelope-from bounces@nabble.com) Received: from kuber.nabble.com (kuber.nabble.com [216.139.236.158]) by mx1.freebsd.org (Postfix) with ESMTP id D1D068FC13 for ; Thu, 13 Nov 2008 06:35:56 +0000 (UTC) (envelope-from bounces@nabble.com) Received: from isper.nabble.com ([192.168.236.156]) by kuber.nabble.com with esmtp (Exim 4.63) (envelope-from ) id 1L0VWl-0003jc-Kn for freebsd-hardware@freebsd.org; Wed, 12 Nov 2008 22:18:03 -0800 Message-ID: <20475350.post@talk.nabble.com> Date: Wed, 12 Nov 2008 22:18:03 -0800 (PST) From: "weinter.lim" To: freebsd-hardware@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Nabble-From: limguowei@gmail.com Subject: Problems With BroadCom NetXtreme Ethernet and Atheros AR5B91 X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Nov 2008 06:35:57 -0000 I am using FreeBSD 7.1 BETA 2 on my New Acer Aspire 4530 During Boot Dmesg did not detect my Ethernet BroadCom NetXtreme pciconf -lv none5@pci0:8:0:0 Class=0X020000 card=0X014a1025 chip=0X168414e4 rev=0X10 hdr=0X00 vendor = 'Broadcom Corporation' class = network subclass = ethernet My Atheros Wireless also failed to detect none6@pci0:11:0:0 Class=0X028000 card=0X03031a32 chip=0X002a168c rev=0X01 hdr=0X00 vendor = 'Atheros Communication Inc' class = network Thanks -- View this message in context: http://www.nabble.com/Problems-With-BroadCom-NetXtreme-Ethernet-and-Atheros-AR5B91-tp20475350p20475350.html Sent from the freebsd-hardware mailing list archive at Nabble.com. From owner-freebsd-hardware@FreeBSD.ORG Thu Nov 13 06:35:57 2008 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 513B21065689 for ; Thu, 13 Nov 2008 06:35:57 +0000 (UTC) (envelope-from bounces@nabble.com) Received: from kuber.nabble.com (kuber.nabble.com [216.139.236.158]) by mx1.freebsd.org (Postfix) with ESMTP id 22C788FC14 for ; Thu, 13 Nov 2008 06:35:57 +0000 (UTC) (envelope-from bounces@nabble.com) Received: from isper.nabble.com ([192.168.236.156]) by kuber.nabble.com with esmtp (Exim 4.63) (envelope-from ) id 1L0VYu-0003nH-6S for freebsd-hardware@freebsd.org; Wed, 12 Nov 2008 22:20:16 -0800 Message-ID: <20475350.post@talk.nabble.com> Date: Wed, 12 Nov 2008 22:20:16 -0800 (PST) From: "weinter.lim" To: freebsd-hardware@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Nabble-From: limguowei@gmail.com Subject: Problems With BroadCom NetXtreme Ethernet and Atheros AR5B91 X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Nov 2008 06:35:57 -0000 I am using FreeBSD 7.1 BETA 2 on my New Acer Aspire 4530 The motherboard belong to Nvidia NForce9 series with the integrated GPU 9100M G During Boot Dmesg did not detect my Ethernet BroadCom NetXtreme pciconf -lv none5@pci0:8:0:0 Class=0X020000 card=0X014a1025 chip=0X168414e4 rev=0X10 hdr=0X00 vendor = 'Broadcom Corporation' class = network subclass = ethernet My Atheros Wireless also failed to detect none6@pci0:11:0:0 Class=0X028000 card=0X03031a32 chip=0X002a168c rev=0X01 hdr=0X00 vendor = 'Atheros Communication Inc' class = network Thanks -- View this message in context: http://www.nabble.com/Problems-With-BroadCom-NetXtreme-Ethernet-and-Atheros-AR5B91-tp20475350p20475350.html Sent from the freebsd-hardware mailing list archive at Nabble.com. From owner-freebsd-hardware@FreeBSD.ORG Thu Nov 13 07:08:19 2008 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0C19D106567A for ; Thu, 13 Nov 2008 07:08:19 +0000 (UTC) (envelope-from won.derick@yahoo.com) Received: from n9.bullet.mail.mud.yahoo.com (n9.bullet.mail.mud.yahoo.com [209.191.86.157]) by mx1.freebsd.org (Postfix) with SMTP id AE9488FC0C for ; Thu, 13 Nov 2008 07:08:18 +0000 (UTC) (envelope-from won.derick@yahoo.com) Received: from [68.142.194.244] by n9.bullet.mail.mud.yahoo.com with NNFMP; 13 Nov 2008 06:55:08 -0000 Received: from [68.142.201.253] by t2.bullet.mud.yahoo.com with NNFMP; 13 Nov 2008 06:55:08 -0000 Received: from [127.0.0.1] by omp414.mail.mud.yahoo.com with NNFMP; 13 Nov 2008 06:55:08 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 109904.48928.bm@omp414.mail.mud.yahoo.com Received: (qmail 61472 invoked by uid 60001); 13 Nov 2008 06:55:07 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:MIME-Version:Content-Type:Message-ID; b=FQr+MubUwhV8Ydetd5Z+s0jQtLEzMoJ23FuLDFrEmxCQ2+vwa4H7A2rBXsQjhecrwHieL4w7q5xo/E21HYsDuwiaiOuj6JdnPmIWTdSt/V2WpIoO5S8b64ef+Ee1bfhgw9HHb+XCWQ49RABK/V73m1kG9XEVWbcYJEPQMuLcfsI=; X-YMail-OSG: d15lJLkVM1kNgom_Jeao4RtJApbHzwWTgS8MWkdbcY7Wlt3AaRstvKfHe4nJZ0U6OPs.iFMdJhhRSOItB6KgB9oWOUt8j9HFYVmPQIhSHj_sO.BD.EHclmaY.7knzzMDukY9xA-- Received: from [58.71.34.137] by web45815.mail.sp1.yahoo.com via HTTP; Wed, 12 Nov 2008 22:55:06 PST X-Mailer: YahooMailRC/1155.20 YahooMailWebService/0.7.260.1 Date: Wed, 12 Nov 2008 22:55:06 -0800 (PST) From: Won De Erick To: freebsd-hardware@freebsd.org MIME-Version: 1.0 Message-ID: <704830.24415.qm@web45815.mail.sp1.yahoo.com> Content-Type: text/plain; charset=us-ascii X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: IRQ31 and IRQ32 on HPDL585 running FreeBSD 7.0 are consuming HIGH CPU usage X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Nov 2008 07:08:19 -0000 Hello, I am conducting a CPU utilization testing with my box(HP DL 585 running FreeBSD 7.0), and come up with the results below: 13 root 1 171 ki31 0K 16K CPU13 d 265:35 100.00% idle: cpu13 18 root 1 171 ki31 0K 16K CPU8 8 265:34 100.00% idle: cpu8 22 root 1 171 ki31 0K 16K CPU4 4 265:34 100.00% idle: cpu4 12 root 1 171 ki31 0K 16K CPU14 e 265:34 100.00% idle: cpu14 23 root 1 171 ki31 0K 16K CPU3 3 265:30 100.00% idle: cpu3 24 root 1 171 ki31 0K 16K CPU2 2 265:21 100.00% idle: cpu2 11 root 1 171 ki31 0K 16K CPU15 f 265:04 100.00% idle: cpu15 14 root 1 171 ki31 0K 16K CPU12 c 264:42 100.00% idle: cpu12 25 root 1 171 ki31 0K 16K CPU1 1 264:29 100.00% idle: cpu1 17 root 1 171 ki31 0K 16K CPU9 9 263:29 100.00% idle: cpu9 21 root 1 171 ki31 0K 16K CPU5 5 261:47 100.00% idle: cpu5 20 root 1 171 ki31 0K 16K CPU6 6 260:56 100.00% idle: cpu6 52 root 1 -68 - 0K 16K CPU11 b 123:53 100.00% irq32: bce1 51 root 1 -68 - 0K 16K CPU10 a 119:28 89.06% irq31: bce0 19 root 1 171 ki31 0K 16K CPU7 7 244:22 85.79% idle: cpu7 26 root 1 171 ki31 0K 16K RUN 0 216:13 54.79% idle: cpu0 27 root 1 -32 - 0K 16K *Giant 0 49:02 48.19% swi4: clock sio 45 root 1 -64 - 0K 16K WAIT 7 21:14 21.29% irq17: uhci0 16 root 1 171 ki31 0K 16K RUN a 146:07 11.08% idle: cpu10 47 root 1 -64 - 0K 16K WAIT 6 4:39 2.88% irq16: ciss0 15 root 1 171 ki31 0K 16K RUN b 141:43 1.17% idle: cpu11 30 root 1 -16 - 0K 16K - f 0:33 0.00% yarrow 61 root 1 20 - 0K 16K syncer e 0:08 0.00% syncer irq31 and irq32 are consuming high CPU usage, which i think the cause of hard reset. I read about the MSI interrupt handling for FreeBSD, and I don't know if there's a way to split the interrupt request for each bce's Rx and Tx, which means a total of four IRQs, and eventually four cores (or 4 CPU) for the transactions. With this way, the IDLE processors would be utilized. Other ways how I can maximize the use of the IDLE CPUs? Thanks, Won From owner-freebsd-hardware@FreeBSD.ORG Thu Nov 13 07:26:11 2008 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8B278106564A for ; Thu, 13 Nov 2008 07:26:11 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from QMTA03.emeryville.ca.mail.comcast.net (qmta03.emeryville.ca.mail.comcast.net [76.96.30.32]) by mx1.freebsd.org (Postfix) with ESMTP id 6B8E18FC16 for ; Thu, 13 Nov 2008 07:26:11 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from OMTA08.emeryville.ca.mail.comcast.net ([76.96.30.12]) by QMTA03.emeryville.ca.mail.comcast.net with comcast id eX8A1a00V0FhH24A3XSBsR; Thu, 13 Nov 2008 07:26:11 +0000 Received: from koitsu.dyndns.org ([69.181.141.110]) by OMTA08.emeryville.ca.mail.comcast.net with comcast id eXSA1a0062P6wsM8UXSA2x; Thu, 13 Nov 2008 07:26:10 +0000 X-Authority-Analysis: v=1.0 c=1 a=nlX-cntIBRQA:10 a=z-gTsQe774kA:10 a=QycZ5dHgAAAA:8 a=AkCrK_DtZzFWMbdHHEEA:9 a=zzzim5LevNrzMmLGI0tZsdoqoqcA:4 a=EoioJ0NPDVgA:10 a=LY0hPdMaydYA:10 Received: by icarus.home.lan (Postfix, from userid 1000) id 3FFB25C19; Wed, 12 Nov 2008 23:26:10 -0800 (PST) Date: Wed, 12 Nov 2008 23:26:10 -0800 From: Jeremy Chadwick To: Won De Erick Message-ID: <20081113072610.GA13698@icarus.home.lan> References: <704830.24415.qm@web45815.mail.sp1.yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <704830.24415.qm@web45815.mail.sp1.yahoo.com> User-Agent: Mutt/1.5.18 (2008-05-17) Cc: freebsd-hardware@freebsd.org Subject: Re: IRQ31 and IRQ32 on HPDL585 running FreeBSD 7.0 are consuming HIGH CPU usage X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Nov 2008 07:26:11 -0000 On Wed, Nov 12, 2008 at 10:55:06PM -0800, Won De Erick wrote: > Hello, Nobody can read the pasted output you've sent. Your Email client is hard-wrapping long lines. Please re-send the data with this feature disabled. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From owner-freebsd-hardware@FreeBSD.ORG Thu Nov 13 07:30:30 2008 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0E15F106567C; Thu, 13 Nov 2008 07:30:30 +0000 (UTC) (envelope-from freebsd@sopwith.solgatos.com) Received: from sopwith.solgatos.com (pool-71-182-69-106.ptldor.fios.verizon.net [71.182.69.106]) by mx1.freebsd.org (Postfix) with ESMTP id 63B708FC12; Thu, 13 Nov 2008 07:30:29 +0000 (UTC) (envelope-from freebsd@sopwith.solgatos.com) Received: by sopwith.solgatos.com (Postfix, from userid 66) id F1EDA3F22; Wed, 12 Nov 2008 22:46:48 -0800 (PST) Received: from localhost by sopwith.solgatos.com (8.8.8/6.24) id GAA26763; Thu, 13 Nov 2008 06:57:58 GMT Message-Id: <200811130657.GAA26763@sopwith.solgatos.com> To: freebsd-fs@freebsd.org, freebsd-hardware@freebsd.org In-reply-to: Your message of "Thu, 13 Nov 2008 15:46:32 +1000." <491BBF38.9010908@dannysplace.net> Date: Wed, 12 Nov 2008 22:57:58 +0000 From: Dieter Cc: Subject: Re: Areca vs. ZFS performance testing. X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Nov 2008 07:30:30 -0000 >> For the array(s) >> 9 x ST31000340AS 1tb disks >> 1 x ST31000333AS 1tb disk (trying to swap this for a ST31000340AS) > There seems to be little difference between enabling and disabling the > disk cache on the Areca. This leads me to two conclusions: > 1. Disabling the write cache does nothing on Seagate drives. > 2. IO to the drives is so slow that a write cache is irrelevant. I have a couple of the ST31000340AS 1TB disks as well as older lower capacity Seagates, and turning the write cache on/off makes a MASSIVE (roughly 10:1) difference in write speed. Jeremy reports "about 13%" with Seagate ST3120026AS: http://lists.freebsd.org/pipermail/freebsd-hardware/2008-October/005450.html Perhaps there is something about the Areca or the testing? Is the write cache really getting turned on/off? You're getting about 2-3x the speed I'd expect if the write cache were off, so maybe it is still on but there is a bottleneck elsewhere? Have you tried a simple test with /dev/zero and dd to a raw drive to eliminate the effects of the filesystem? From owner-freebsd-hardware@FreeBSD.ORG Thu Nov 13 07:38:17 2008 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 39C821065676 for ; Thu, 13 Nov 2008 07:38:17 +0000 (UTC) (envelope-from won.derick@yahoo.com) Received: from n17.bullet.mail.mud.yahoo.com (n17.bullet.mail.mud.yahoo.com [68.142.206.144]) by mx1.freebsd.org (Postfix) with SMTP id F231E8FC1E for ; Thu, 13 Nov 2008 07:38:16 +0000 (UTC) (envelope-from won.derick@yahoo.com) Received: from [209.191.108.96] by n17.bullet.mail.mud.yahoo.com with NNFMP; 13 Nov 2008 07:38:16 -0000 Received: from [68.142.201.66] by t3.bullet.mud.yahoo.com with NNFMP; 13 Nov 2008 07:38:16 -0000 Received: from [127.0.0.1] by omp418.mail.mud.yahoo.com with NNFMP; 13 Nov 2008 07:38:16 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 104794.16964.bm@omp418.mail.mud.yahoo.com Received: (qmail 44017 invoked by uid 60001); 13 Nov 2008 07:38:15 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Mailer:References:Date:From:Subject:To:Cc:MIME-Version:Content-Type:Message-ID; b=5CFaRqQo6ClTj5uW2DwcQ5njb1x0nERCca+XKHf8leAclTdu0vqP6YBnOtX9UYwTxdtlRGsa97q4lpmTtmoxj+DBQnQMRMi/vZQTjFTytu84Shu8HN/ZEQDbQ7K8OfBuTca38/V0gfuY4rDjfYYT0TLkwZztc6GNUcGTRuQBd44=; X-YMail-OSG: 2.TPgEMVM1mPzDHaUuYN3s4oKG0hqjKAN8po5RFTcr3P_JrAuzYzp6gUP8bxz8X59wSMPsHR_nZl0srrfeBzuduzAVtdAmhrXmgquda2FsiHDS9P_fjs7crv5NALCxQyiCYUtLsTfPcv9t6onJ9cSkuMLLQ- Received: from [58.71.34.137] by web45812.mail.sp1.yahoo.com via HTTP; Wed, 12 Nov 2008 23:38:15 PST X-Mailer: YahooMailRC/1155.20 YahooMailWebService/0.7.260.1 References: <704830.24415.qm@web45815.mail.sp1.yahoo.com> <20081113072610.GA13698@icarus.home.lan> Date: Wed, 12 Nov 2008 23:38:15 -0800 (PST) From: Won De Erick To: Jeremy Chadwick MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Message-ID: <576266.41435.qm@web45812.mail.sp1.yahoo.com> Cc: freebsd-hardware@freebsd.org Subject: Re: IRQ31 and IRQ32 on HPDL585 running FreeBSD 7.0 are consuming HIGH CPU usage X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Nov 2008 07:38:17 -0000 resending my previous email. ---------------------------------------------------------------------------------------------------------------------- Hello, I am conducting a CPU utilization testing with my box(HP DL 585 running FreeBSD 7.0), and come up with the results below: 13 root 1 171 ki31 0K 16K CPU13 d 265:35 100.00% idle: cpu13 18 root 1 171 ki31 0K 16K CPU8 8 265:34 100.00% idle: cpu8 22 root 1 171 ki31 0K 16K CPU4 4 265:34 100.00% idle: cpu4 12 root 1 171 ki31 0K 16K CPU14 e 265:34 100.00% idle: cpu14 23 root 1 171 ki31 0K 16K CPU3 3 265:30 100.00% idle: cpu3 24 root 1 171 ki31 0K 16K CPU2 2 265:21 100.00% idle: cpu2 11 root 1 171 ki31 0K 16K CPU15 f 265:04 100.00% idle: cpu15 14 root 1 171 ki31 0K 16K CPU12 c 264:42 100.00% idle: cpu12 25 root 1 171 ki31 0K 16K CPU1 1 264:29 100.00% idle: cpu1 17 root 1 171 ki31 0K 16K CPU9 9 263:29 100.00% idle: cpu9 21 root 1 171 ki31 0K 16K CPU5 5 261:47 100.00% idle: cpu5 20 root 1 171 ki31 0K 16K CPU6 6 260:56 100.00% idle: cpu6 52 root 1 -68 - 0K 16K CPU11 b 123:53 100.00% irq32: bce1 51 root 1 -68 - 0K 16K CPU10 a 119:28 89.06% irq31: bce0 19 root 1 171 ki31 0K 16K CPU7 7 244:22 85.79% idle: cpu7 26 root 1 171 ki31 0K 16K RUN 0 216:13 54.79% idle: cpu0 27 root 1 -32 - 0K 16K *Giant 0 49:02 48.19% swi4: clock sio 45 root 1 -64 - 0K 16K WAIT 7 21:14 21.29% irq17: uhci0 16 root 1 171 ki31 0K 16K RUN a 146:07 11.08% idle: cpu10 47 root 1 -64 - 0K 16K WAIT 6 4:39 2.88% irq16: ciss0 15 root 1 171 ki31 0K 16K RUN b 141:43 1.17% idle: cpu11 30 root 1 -16 - 0K 16K - f 0:33 0.00% yarrow 61 root 1 20 - 0K 16K syncer e 0:08 0.00% syncer irq31 and irq32 are consuming high CPU usage, which i think the cause of hard reset. I read about the MSI interrupt handling for FreeBSD, and I don't know if there's a way to split the interrupt request for each bce's Rx and Tx, which means a total of four IRQs, and eventually four cores (or 4 CPU) for the transactions. With this way, the IDLE processors would be utilized. Other ways how I can maximize the use of the IDLE CPUs? Thanks, Won --------------------------------------------------------------------------------------------------------------------------- ________________________________ From: Jeremy Chadwick To: Won De Erick Cc: freebsd-hardware@freebsd.org Sent: Thursday, November 13, 2008 3:26:10 PM Subject: Re: IRQ31 and IRQ32 on HPDL585 running FreeBSD 7.0 are consuming HIGH CPU usage On Wed, Nov 12, 2008 at 10:55:06PM -0800, Won De Erick wrote: > Hello, Nobody can read the pasted output you've sent. Your Email client is hard-wrapping long lines. Please re-send the data with this feature disabled. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | _______________________________________________ freebsd-hardware@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hardware To unsubscribe, send any mail to "freebsd-hardware-unsubscribe@freebsd.org" From owner-freebsd-hardware@FreeBSD.ORG Thu Nov 13 07:43:03 2008 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 47B56106567D for ; Thu, 13 Nov 2008 07:43:03 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from QMTA09.emeryville.ca.mail.comcast.net (qmta09.emeryville.ca.mail.comcast.net [76.96.30.96]) by mx1.freebsd.org (Postfix) with ESMTP id 270408FC17 for ; Thu, 13 Nov 2008 07:43:02 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from OMTA13.emeryville.ca.mail.comcast.net ([76.96.30.52]) by QMTA09.emeryville.ca.mail.comcast.net with comcast id eXKa1a00917UAYkA9Xj2dL; Thu, 13 Nov 2008 07:43:02 +0000 Received: from koitsu.dyndns.org ([69.181.141.110]) by OMTA13.emeryville.ca.mail.comcast.net with comcast id eXj21a0012P6wsM8ZXj26s; Thu, 13 Nov 2008 07:43:02 +0000 X-Authority-Analysis: v=1.0 c=1 a=aQgbMQmz5TEA:10 a=qMCG-Xc8eBMA:10 a=6I5d2MoRAAAA:8 a=QycZ5dHgAAAA:8 a=ggi0_wdoAqmuBgJFduwA:9 a=_n-Wm4Mgt0fDCR6WTWreOFjnpmsA:4 a=EoioJ0NPDVgA:10 a=LY0hPdMaydYA:10 Received: by icarus.home.lan (Postfix, from userid 1000) id E21665C19; Wed, 12 Nov 2008 23:43:01 -0800 (PST) Date: Wed, 12 Nov 2008 23:43:01 -0800 From: Jeremy Chadwick To: Dieter Message-ID: <20081113074301.GA13938@icarus.home.lan> References: <491BBF38.9010908@dannysplace.net> <200811130657.GAA26763@sopwith.solgatos.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200811130657.GAA26763@sopwith.solgatos.com> User-Agent: Mutt/1.5.18 (2008-05-17) Cc: freebsd-fs@freebsd.org, freebsd-hardware@freebsd.org Subject: Re: Areca vs. ZFS performance testing. X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Nov 2008 07:43:03 -0000 On Wed, Nov 12, 2008 at 10:57:58PM +0000, Dieter wrote: > >> For the array(s) > >> 9 x ST31000340AS 1tb disks > >> 1 x ST31000333AS 1tb disk (trying to swap this for a ST31000340AS) > > > There seems to be little difference between enabling and disabling the > > disk cache on the Areca. This leads me to two conclusions: > > 1. Disabling the write cache does nothing on Seagate drives. > > 2. IO to the drives is so slow that a write cache is irrelevant. > > I have a couple of the ST31000340AS 1TB disks as well as older lower capacity > Seagates, and turning the write cache on/off makes a MASSIVE (roughly 10:1) > difference in write speed. > > Jeremy reports "about 13%" with Seagate ST3120026AS: > http://lists.freebsd.org/pipermail/freebsd-hardware/2008-October/005450.html > > Perhaps there is something about the Areca or the testing? Is the write cache > really getting turned on/off? The Areca controller he has can do caching of its own (it has 256MBytes of cache). Meaning, if you disable write cache on the disks (but not the Areca controller itself), all of the caching being done is purely controller-based. The actual disk writes between the controller and the disk will, of course, be "slow" -- but between the OS and the controller, things should appear fast. Let me outline the 4 test scenarios (I thought I did this in my original mail to Danny, but I believe I also said "don't get caught up in excessive granularity because it'll just confuse people now" -- case in point): - Areca cache disabled, disk write cache enabled - Areca cache disabled, disk write cache disabled - Areca cache enabled, disk write cache enabled - Areca cache enabled, disk write cache disabled [**] As I understand it, Danny performed the tests with the [**] configuration. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From owner-freebsd-hardware@FreeBSD.ORG Thu Nov 13 07:46:04 2008 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 774001065689 for ; Thu, 13 Nov 2008 07:46:04 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from QMTA10.westchester.pa.mail.comcast.net (qmta10.westchester.pa.mail.comcast.net [76.96.62.17]) by mx1.freebsd.org (Postfix) with ESMTP id 028F18FC12 for ; Thu, 13 Nov 2008 07:46:03 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from OMTA04.westchester.pa.mail.comcast.net ([76.96.62.35]) by QMTA10.westchester.pa.mail.comcast.net with comcast id eXj61a00B0ldTLk5AXlxst; Thu, 13 Nov 2008 07:45:57 +0000 Received: from koitsu.dyndns.org ([69.181.141.110]) by OMTA04.westchester.pa.mail.comcast.net with comcast id eXm21a0052P6wsM3QXm25a; Thu, 13 Nov 2008 07:46:03 +0000 X-Authority-Analysis: v=1.0 c=1 a=nlX-cntIBRQA:10 a=z-gTsQe774kA:10 a=6I5d2MoRAAAA:8 a=QycZ5dHgAAAA:8 a=K3RAw2WYEhym6H3HUHgA:9 a=HwoWUP2uEPxGZe1KcU7VXgC945YA:4 a=EoioJ0NPDVgA:10 a=LY0hPdMaydYA:10 Received: by icarus.home.lan (Postfix, from userid 1000) id 3556D5C19; Wed, 12 Nov 2008 23:46:02 -0800 (PST) Date: Wed, 12 Nov 2008 23:46:02 -0800 From: Jeremy Chadwick To: Won De Erick Message-ID: <20081113074602.GB13938@icarus.home.lan> References: <704830.24415.qm@web45815.mail.sp1.yahoo.com> <20081113072610.GA13698@icarus.home.lan> <576266.41435.qm@web45812.mail.sp1.yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <576266.41435.qm@web45812.mail.sp1.yahoo.com> User-Agent: Mutt/1.5.18 (2008-05-17) Cc: freebsd-hardware@freebsd.org Subject: Re: IRQ31 and IRQ32 on HPDL585 running FreeBSD 7.0 are consuming HIGH CPU usage X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Nov 2008 07:46:04 -0000 On Wed, Nov 12, 2008 at 11:38:15PM -0800, Won De Erick wrote: > I am conducting a CPU utilization testing with my box(HP DL 585 running FreeBSD 7.0), and come up with the results below: > > 52 root 1 -68 - 0K 16K CPU11 b 123:53 100.00% irq32: bce1 > 51 root 1 -68 - 0K 16K CPU10 a 119:28 89.06% irq31: bce0 > > irq31 and irq32 are consuming high CPU usage, which i think the cause of hard reset. There was a ***major*** bce(4) cleanup that just happened. Your 7.0 box will not have these changes. Please upgrade your box to RELENG_7 (a.k.a. 7.1-PRERELEASE), csup'd recently (today preferably), and try your tests again: http://lists.freebsd.org/pipermail/freebsd-stable/2008-November/046482.html -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From owner-freebsd-hardware@FreeBSD.ORG Thu Nov 13 08:19:38 2008 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 614FB106564A for ; Thu, 13 Nov 2008 08:19:38 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from QMTA02.westchester.pa.mail.comcast.net (qmta02.westchester.pa.mail.comcast.net [76.96.62.24]) by mx1.freebsd.org (Postfix) with ESMTP id 07FBF8FC1D for ; Thu, 13 Nov 2008 08:19:37 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from OMTA11.westchester.pa.mail.comcast.net ([76.96.62.36]) by QMTA02.westchester.pa.mail.comcast.net with comcast id eYH31a00B0mv7h052YKLoF; Thu, 13 Nov 2008 08:19:20 +0000 Received: from koitsu.dyndns.org ([69.181.141.110]) by OMTA11.westchester.pa.mail.comcast.net with comcast id eYKc1a0062P6wsM3XYKcHn; Thu, 13 Nov 2008 08:19:37 +0000 X-Authority-Analysis: v=1.0 c=1 a=nlX-cntIBRQA:10 a=z-gTsQe774kA:10 a=QycZ5dHgAAAA:8 a=iLG4jCQ8vDJgY5MNXbQA:9 a=ZqKsnU7jZrpszV8LU_49QTTQ9mAA:4 a=EoioJ0NPDVgA:10 a=LY0hPdMaydYA:10 Received: by icarus.home.lan (Postfix, from userid 1000) id 5467B5C19; Thu, 13 Nov 2008 00:19:36 -0800 (PST) Date: Thu, 13 Nov 2008 00:19:36 -0800 From: Jeremy Chadwick To: Won De Erick Message-ID: <20081113081936.GA14779@icarus.home.lan> References: <704830.24415.qm@web45815.mail.sp1.yahoo.com> <20081113072610.GA13698@icarus.home.lan> <576266.41435.qm@web45812.mail.sp1.yahoo.com> <20081113074602.GB13938@icarus.home.lan> <366483.43588.qm@web45807.mail.sp1.yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <366483.43588.qm@web45807.mail.sp1.yahoo.com> User-Agent: Mutt/1.5.18 (2008-05-17) Cc: freebsd-hardware@freebsd.org Subject: Re: IRQ31 and IRQ32 on HPDL585 running FreeBSD 7.0 are consuming HIGH CPU usage X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Nov 2008 08:19:38 -0000 On Thu, Nov 13, 2008 at 12:07:37AM -0800, Won De Erick wrote: > Noted on this, I will update you through this thread. > > However is there any possibility of the following: > > > I don't know if there's a way to split the interrupt request for each bce's Rx and Tx, > > which means a total of four IRQs, and eventually four cores (or 4 CPUs) > > for the transactions. With this way, the IDLE processors would be utilized. > > What I mean here is, for the two interfaces: > > one IRQ for bce0 Rx > one IRQ for bce0 Tx > one IRQ for bce1 Rx > one IRQ for bce1 Tx I can't even begin to imagine how this would be possible on any NIC. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From owner-freebsd-hardware@FreeBSD.ORG Thu Nov 13 08:21:25 2008 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 35F8B1065686 for ; Thu, 13 Nov 2008 08:21:25 +0000 (UTC) (envelope-from won.derick@yahoo.com) Received: from n67.bullet.mail.sp1.yahoo.com (n67.bullet.mail.sp1.yahoo.com [98.136.44.47]) by mx1.freebsd.org (Postfix) with SMTP id 140568FC1C for ; Thu, 13 Nov 2008 08:21:25 +0000 (UTC) (envelope-from won.derick@yahoo.com) Received: from [69.147.65.150] by n67.bullet.mail.sp1.yahoo.com with NNFMP; 13 Nov 2008 08:07:37 -0000 Received: from [69.147.65.164] by t7.bullet.mail.sp1.yahoo.com with NNFMP; 13 Nov 2008 08:07:37 -0000 Received: from [127.0.0.1] by omp409.mail.sp1.yahoo.com with NNFMP; 13 Nov 2008 08:07:37 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 515402.30127.bm@omp409.mail.sp1.yahoo.com Received: (qmail 47331 invoked by uid 60001); 13 Nov 2008 08:07:37 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Mailer:References:Date:From:Subject:To:Cc:MIME-Version:Content-Type:Message-ID; b=0YVXBzWy8KdjfyUyk58JbM4kx9go81dzGDIjp4WI5Zm45heOahXVQo0RXggyFMVNzErT+rOKV+j+0C6IBCnawXWcanvstXG52LgeOXuF/4lE7WbkhCRFI1GbDsylLqE/bH9UKEnYSGETZauJGXixpV1DiIYN8FnyP1LL70OQaEQ=; X-YMail-OSG: V1oh.3gVM1mfCTFI_k0xucdcgEzb47lGSUfxyoYcd4WETrTkK1snTHjNakWi5SWzH1cPWiliTuNCW.caBSrDICJI5KnPRRHmins.RXCZHVuXgvOgBYlxcLLXzUyFqmE9OVpIDHvCI7B6.JaeEU8dp1iLNXk- Received: from [58.71.34.137] by web45807.mail.sp1.yahoo.com via HTTP; Thu, 13 Nov 2008 00:07:37 PST X-Mailer: YahooMailRC/1155.20 YahooMailWebService/0.7.260.1 References: <704830.24415.qm@web45815.mail.sp1.yahoo.com> <20081113072610.GA13698@icarus.home.lan> <576266.41435.qm@web45812.mail.sp1.yahoo.com> <20081113074602.GB13938@icarus.home.lan> Date: Thu, 13 Nov 2008 00:07:37 -0800 (PST) From: Won De Erick To: Jeremy Chadwick MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Message-ID: <366483.43588.qm@web45807.mail.sp1.yahoo.com> Cc: freebsd-hardware@freebsd.org Subject: Re: IRQ31 and IRQ32 on HPDL585 running FreeBSD 7.0 are consuming HIGH CPU usage X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Nov 2008 08:21:25 -0000 Noted on this, I will update you through this thread. However is there any possibility of the following: > I don't know if there's a way to split the interrupt request for each bce's Rx and Tx, > which means a total of four IRQs, and eventually four cores (or 4 CPUs) > for the transactions. With this way, the IDLE processors would be utilized. What I mean here is, for the two interfaces: one IRQ for bce0 Rx one IRQ for bce0 Tx one IRQ for bce1 Rx one IRQ for bce1 Tx Thanks, Won ________________________________ From: Jeremy Chadwick To: Won De Erick Cc: freebsd-hardware@freebsd.org Sent: Thursday, November 13, 2008 3:46:02 PM Subject: Re: IRQ31 and IRQ32 on HPDL585 running FreeBSD 7.0 are consuming HIGH CPU usage On Wed, Nov 12, 2008 at 11:38:15PM -0800, Won De Erick wrote: > I am conducting a CPU utilization testing with my box(HP DL 585 running FreeBSD 7.0), and come up with the results below: > > 52 root 1 -68 - 0K 16K CPU11 b 123:53 100.00% irq32: bce1 > 51 root 1 -68 - 0K 16K CPU10 a 119:28 89.06% irq31: bce0 > > irq31 and irq32 are consuming high CPU usage, which i think the cause of hard reset. There was a ***major*** bce(4) cleanup that just happened. Your 7.0 box will not have these changes. Please upgrade your box to RELENG_7 (a.k.a. 7.1-PRERELEASE), csup'd recently (today preferably), and try your tests again: http://lists.freebsd.org/pipermail/freebsd-stable/2008-November/046482.html -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | _______________________________________________ freebsd-hardware@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hardware To unsubscribe, send any mail to "freebsd-hardware-unsubscribe@freebsd.org" From owner-freebsd-hardware@FreeBSD.ORG Thu Nov 13 08:53:27 2008 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 59A9F1065670; Thu, 13 Nov 2008 08:53:27 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from mail.digiware.nl (www.tegenbosch28.nl [217.21.251.97]) by mx1.freebsd.org (Postfix) with ESMTP id EA2708FC12; Thu, 13 Nov 2008 08:53:26 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from localhost (localhost.digiware.nl [127.0.0.1]) by mail.digiware.nl (Postfix) with ESMTP id 0709F1765B; Thu, 13 Nov 2008 09:33:04 +0100 (CET) X-Virus-Scanned: amavisd-new at digiware.nl Received: from mail.digiware.nl ([127.0.0.1]) by localhost (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YMOew9zOnKSU; Thu, 13 Nov 2008 09:32:52 +0100 (CET) Received: from [212.61.27.67] (opteron [212.61.27.67]) by mail.digiware.nl (Postfix) with ESMTP id BA91A174FF; Thu, 13 Nov 2008 09:32:51 +0100 (CET) Message-ID: <491BE632.1020801@IMAP> Date: Thu, 13 Nov 2008 09:32:50 +0100 From: Willem Jan Withagen User-Agent: Thunderbird 2.0.0.17 (X11/20080925) MIME-Version: 1.0 To: Danny Carroll References: <490A782F.9060406@dannysplace.net> <20081031033208.GA21220@icarus.home.lan> <490A849C.7030009@dannysplace.net> <20081031043412.GA22289@icarus.home.lan> <490A8FAD.8060009@dannysplace.net> <491BBF38.9010908@dannysplace.net> In-Reply-To: <491BBF38.9010908@dannysplace.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org, Jeremy Chadwick , fbsd@dannysplace.net, freebsd-hardware@freebsd.org Subject: Re: Areca vs. ZFS performance testing. X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Nov 2008 08:53:27 -0000 Danny Carroll wrote: > Danny Carroll wrote: >> Jeremy Chadwick wrote: >>> I'd like to see the performance difference between these scenarios: >>> >>> - Memory cache enabled on Areca, write caching enabled on disks >>> - Memory cache enabled on Areca, write caching disabled on disks >>> - Memory cache disabled on Areca, write caching enabled on disks >>> - Memory cache disabled on Areca, write caching disabled on disks >>> > > > The initial results for a ICH9 vs Areca in JBod mode can be found here: > http://www.dannysplace.net/ZFS-JBODTests.html Just as a polite question, since I'm very much in favor doing benchmarking and do appreciate these kinds of test. You might want to add an introductory page to your results describing how you setup the test: Details of the hardware Details of the disk setup possible version and options with bonnie The script you used.... This would allow others to redo your experiment and try to figure out why their numbers are different. --WjW From owner-freebsd-hardware@FreeBSD.ORG Thu Nov 13 09:34:02 2008 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4A6861065672 for ; Thu, 13 Nov 2008 09:34:02 +0000 (UTC) (envelope-from won.derick@yahoo.com) Received: from n70.bullet.mail.sp1.yahoo.com (n70.bullet.mail.sp1.yahoo.com [98.136.44.38]) by mx1.freebsd.org (Postfix) with SMTP id 0A8D98FC13 for ; Thu, 13 Nov 2008 09:34:02 +0000 (UTC) (envelope-from won.derick@yahoo.com) Received: from [69.147.65.150] by n70.bullet.mail.sp1.yahoo.com with NNFMP; 13 Nov 2008 09:34:02 -0000 Received: from [69.147.84.116] by t7.bullet.mail.sp1.yahoo.com with NNFMP; 13 Nov 2008 09:34:01 -0000 Received: from [127.0.0.1] by omp208.mail.sp1.yahoo.com with NNFMP; 13 Nov 2008 09:34:01 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 967560.8973.bm@omp208.mail.sp1.yahoo.com Received: (qmail 79921 invoked by uid 60001); 13 Nov 2008 09:34:01 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Mailer:References:Date:From:Subject:To:Cc:MIME-Version:Content-Type:Message-ID; b=XribIfV+jTG7A7j/NXI9Ps30gW2vLG6dtKZLhF3gMfrxnf10ZzPXH5EpHM/YZQ3Ar0lrhUvsSyzcWvo2f5dYJ/XWDiUkoE/+XjDDA6XfDPFHzBbcIzFvFI2q5bmfallVkYi1op2f9VRGaeBDm0AAfPP5qybTD34q50j5kSJ82j4=; X-YMail-OSG: wcYulVgVM1nOhSh5MJAW4q7UDMxBcaWST6tR6KSgPcnYnoXtqmojjUlQ4uy73LRwxxQjxpfNAfQd7l84.lwgNWBLhvIb3erEWZRJ6qu4oAuTTFOKkFV0Av4tcf74EirJcGIzAQ-- Received: from [58.71.34.137] by web45804.mail.sp1.yahoo.com via HTTP; Thu, 13 Nov 2008 01:34:01 PST X-Mailer: YahooMailRC/1155.20 YahooMailWebService/0.7.260.1 References: <704830.24415.qm@web45815.mail.sp1.yahoo.com> <20081113072610.GA13698@icarus.home.lan> <576266.41435.qm@web45812.mail.sp1.yahoo.com> <20081113074602.GB13938@icarus.home.lan> <366483.43588.qm@web45807.mail.sp1.yahoo.com> <20081113081936.GA14779@icarus.home.lan> Date: Thu, 13 Nov 2008 01:34:01 -0800 (PST) From: Won De Erick To: Jeremy Chadwick MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Message-ID: <527634.76455.qm@web45804.mail.sp1.yahoo.com> Cc: freebsd-hardware@freebsd.org Subject: Re: IRQ31 and IRQ32 on HPDL585 running FreeBSD 7.0 are consuming HIGH CPU usage X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Nov 2008 09:34:02 -0000 Jeremy Chadwick wrote: >> On Thu, Nov 13, 2008 at 12:07:37AM -0800, Won De Erick wrote: >> Noted on this, I will update you through this thread. >> >> However is there any possibility of the following: >> >> > I don't know if there's a way to split the interrupt request for each bce's Rx and Tx, >> > which means a total of four IRQs, and eventually four cores (or 4 CPUs) >> > for the transactions. With this way, the IDLE processors would be utilized. >> >> What I mean here is, for the two interfaces: >> >> one IRQ for bce0 Rx >> one IRQ for bce0 Tx >> one IRQ for bce1 Rx >> one IRQ for bce1 Tx > I can't even begin to imagine how this would be possible on any NIC. Yeah, but I can't also imagine how beneficial this idea when implemented. From owner-freebsd-hardware@FreeBSD.ORG Thu Nov 13 09:37:09 2008 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4A3531065690 for ; Thu, 13 Nov 2008 09:37:09 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from QMTA03.emeryville.ca.mail.comcast.net (qmta03.emeryville.ca.mail.comcast.net [76.96.30.32]) by mx1.freebsd.org (Postfix) with ESMTP id 2C4118FC13 for ; Thu, 13 Nov 2008 09:37:08 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from OMTA12.emeryville.ca.mail.comcast.net ([76.96.30.44]) by QMTA03.emeryville.ca.mail.comcast.net with comcast id eZa41a0050x6nqcA3Zd8PB; Thu, 13 Nov 2008 09:37:08 +0000 Received: from koitsu.dyndns.org ([69.181.141.110]) by OMTA12.emeryville.ca.mail.comcast.net with comcast id eZd81a0012P6wsM8YZd8W2; Thu, 13 Nov 2008 09:37:08 +0000 X-Authority-Analysis: v=1.0 c=1 a=nlX-cntIBRQA:10 a=z-gTsQe774kA:10 a=QycZ5dHgAAAA:8 a=3OPnk5vVGRRzsAaJBpYA:9 a=DNucRc215qdeSwTDK7_kNCUOyZ4A:4 a=EoioJ0NPDVgA:10 a=LY0hPdMaydYA:10 Received: by icarus.home.lan (Postfix, from userid 1000) id C61235C19; Thu, 13 Nov 2008 01:37:07 -0800 (PST) Date: Thu, 13 Nov 2008 01:37:07 -0800 From: Jeremy Chadwick To: Won De Erick Message-ID: <20081113093707.GA16322@icarus.home.lan> References: <704830.24415.qm@web45815.mail.sp1.yahoo.com> <20081113072610.GA13698@icarus.home.lan> <576266.41435.qm@web45812.mail.sp1.yahoo.com> <20081113074602.GB13938@icarus.home.lan> <366483.43588.qm@web45807.mail.sp1.yahoo.com> <20081113081936.GA14779@icarus.home.lan> <527634.76455.qm@web45804.mail.sp1.yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <527634.76455.qm@web45804.mail.sp1.yahoo.com> User-Agent: Mutt/1.5.18 (2008-05-17) Cc: freebsd-hardware@freebsd.org Subject: Re: IRQ31 and IRQ32 on HPDL585 running FreeBSD 7.0 are consuming HIGH CPU usage X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Nov 2008 09:37:09 -0000 On Thu, Nov 13, 2008 at 01:34:01AM -0800, Won De Erick wrote: > Jeremy Chadwick wrote: > > >> On Thu, Nov 13, 2008 at 12:07:37AM -0800, Won De Erick wrote: > >> Noted on this, I will update you through this thread. > >> > >> However is there any possibility of the following: > >> > >> > I don't know if there's a way to split the interrupt request for each bce's Rx and Tx, > >> > which means a total of four IRQs, and eventually four cores (or 4 CPUs) > >> > for the transactions. With this way, the IDLE processors would be utilized. > >> > >> What I mean here is, for the two interfaces: > >> > >> one IRQ for bce0 Rx > >> one IRQ for bce0 Tx > >> one IRQ for bce1 Rx > >> one IRQ for bce1 Tx > > > I can't even begin to imagine how this would be possible on any NIC. > > Yeah, but I can't also imagine how beneficial this idea when implemented. Right -- me either. What you just said begs the question. :-) -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From owner-freebsd-hardware@FreeBSD.ORG Thu Nov 13 10:21:58 2008 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F3EF71065676; Thu, 13 Nov 2008 10:21:57 +0000 (UTC) (envelope-from rmaglasang@infoweapons.com) Received: from infoweapons.com (mail0.infoweapons.org [204.2.248.50]) by mx1.freebsd.org (Postfix) with ESMTP id 9F1508FC18; Thu, 13 Nov 2008 10:21:57 +0000 (UTC) (envelope-from rmaglasang@infoweapons.com) Received: from ([58.71.34.146]) by mail0.infoweapons.com with ESMTP id 4321444.1329412; Thu, 13 Nov 2008 05:06:32 -0500 Received: from [10.3.1.41] ([10.3.1.41]) by cebexch01.cebu.infoweapons.com over TLS secured channel with Microsoft SMTPSVC(6.0.3790.3959); Thu, 13 Nov 2008 18:06:32 +0800 Message-ID: <491BFB68.7050405@infoweapons.com> Date: Thu, 13 Nov 2008 18:03:20 +0800 From: "Ronnel P. Maglasang" User-Agent: Thunderbird 1.5 (X11/20060613) MIME-Version: 1.0 To: freebsd-hardware@freebsd.org, freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 13 Nov 2008 10:06:32.0107 (UTC) FILETIME=[80CC57B0:01C94577] Cc: Subject: assigning interrupts X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Nov 2008 10:21:58 -0000 Hi All, Is there a way to explicitly assign an interrupt of a device? I'm running on 6.3 and the two NICs share the same interrupt. Obviously this will affect the performance if the NICs are exposed to heavy network traffic. # vmstat -i interrupt total rate irq11: em0 vr0+ 1081099 77 Total 16958562 1222 Looking at the driver's code, I have the initial though that this is the place where I can modify. -- adapter->res_interrupt = bus_alloc_resource_any(dev, SYS_RES_IRQ, &rid, RF_SHAREABLE | RF_ACTIVE); -- I've tried changing RF_SHAREABLE to RF_ALLOCATED or other values but still could not get the desired result and worst the device fail to initialize. Is this possible in 6.3? Thanks, Ronnel From owner-freebsd-hardware@FreeBSD.ORG Thu Nov 13 10:40:56 2008 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7842F1065690 for ; Thu, 13 Nov 2008 10:40:56 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from QMTA05.westchester.pa.mail.comcast.net (qmta05.westchester.pa.mail.comcast.net [76.96.62.48]) by mx1.freebsd.org (Postfix) with ESMTP id 261188FC0C for ; Thu, 13 Nov 2008 10:40:55 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from OMTA01.westchester.pa.mail.comcast.net ([76.96.62.11]) by QMTA05.westchester.pa.mail.comcast.net with comcast id eaE81a00F0EZKEL55agv0C; Thu, 13 Nov 2008 10:40:55 +0000 Received: from koitsu.dyndns.org ([69.181.141.110]) by OMTA01.westchester.pa.mail.comcast.net with comcast id eagu1a0052P6wsM3MaguPP; Thu, 13 Nov 2008 10:40:55 +0000 X-Authority-Analysis: v=1.0 c=1 a=QycZ5dHgAAAA:8 a=BkQB1dUM75_zf09Ah6kA:9 a=J1uY52gd92DIS6vZOeIdoHJOzmIA:4 a=EoioJ0NPDVgA:10 a=LY0hPdMaydYA:10 Received: by icarus.home.lan (Postfix, from userid 1000) id 4B0695C19; Thu, 13 Nov 2008 02:40:54 -0800 (PST) Date: Thu, 13 Nov 2008 02:40:54 -0800 From: Jeremy Chadwick To: "Ronnel P. Maglasang" Message-ID: <20081113104054.GA17501@icarus.home.lan> References: <491BFB68.7050405@infoweapons.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <491BFB68.7050405@infoweapons.com> User-Agent: Mutt/1.5.18 (2008-05-17) Cc: freebsd-hackers@freebsd.org, freebsd-hardware@freebsd.org Subject: Re: assigning interrupts X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Nov 2008 10:40:56 -0000 On Thu, Nov 13, 2008 at 06:03:20PM +0800, Ronnel P. Maglasang wrote: > Hi All, > > Is there a way to explicitly assign an interrupt > of a device? I'm running on 6.3 and the two NICs > share the same interrupt. Obviously this will affect > the performance if the NICs are exposed to heavy network > traffic. > > # vmstat -i > interrupt total rate > > irq11: em0 vr0+ 1081099 77 > > Total 16958562 1222 > > > Looking at the driver's code, I have the initial though > that this is the place where I can modify. This is the responsibility of the BIOS or ACPI configuration. There is no way to do this via OS software, as far as I know. Try looking at the motherboard manual for what PCI levels (A/B/C/D) share IRQs with what slot, then move cards around. Otherwise, consider purchasing a motherboard that has an APIC (this is not a typo) increasing the IRQ count to 256. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From owner-freebsd-hardware@FreeBSD.ORG Thu Nov 13 10:43:42 2008 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E3A47106564A for ; Thu, 13 Nov 2008 10:43:42 +0000 (UTC) (envelope-from rea-fbsd@codelabs.ru) Received: from 0.mx.codelabs.ru (0.mx.codelabs.ru [144.206.177.45]) by mx1.freebsd.org (Postfix) with ESMTP id 96A0C8FC18 for ; Thu, 13 Nov 2008 10:43:42 +0000 (UTC) (envelope-from rea-fbsd@codelabs.ru) DomainKey-Signature: a=rsa-sha1; q=dns; c=simple; s=one; d=codelabs.ru; h=Received:Date:From:To:Cc:Subject:Message-ID:References:MIME-Version:Content-Type:Content-Disposition:In-Reply-To:Sender; b=MoGjvPHGDBcXzmcdnlIMQ9PtzCw1vR9FV+XU4pIv28RftYG/ClR9Q+q6vtn17EzlufIJFBf8hW3vqC5/DxhEVmx/hoHh2RQ0WzchZR6UOEYgxQq6Z0tzBSrRBJeT+tYU1POrhBUEZpLSIww7R+7zqqSfk5tx0gN7yDL2aOjqc3U=; Received: from void.codelabs.ru (void.codelabs.ru [144.206.177.25]) by 0.mx.codelabs.ru with esmtpsa (TLSv1:AES256-SHA:256) id 1L0ZTu-000Mln-Bm; Thu, 13 Nov 2008 13:31:22 +0300 Date: Thu, 13 Nov 2008 13:31:21 +0300 From: Eygene Ryabinkin To: "Ronnel P. Maglasang" Message-ID: References: <491BFB68.7050405@infoweapons.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="E/DnYTRukya0zdZ1" Content-Disposition: inline In-Reply-To: <491BFB68.7050405@infoweapons.com> Sender: rea-fbsd@codelabs.ru Cc: freebsd-hackers@freebsd.org, freebsd-hardware@freebsd.org Subject: Re: assigning interrupts X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Nov 2008 10:43:43 -0000 --E/DnYTRukya0zdZ1 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Thu, Nov 13, 2008 at 06:03:20PM +0800, Ronnel P. Maglasang wrote: > Is there a way to explicitly assign an interrupt > of a device? What about BIOS? What about physically reshuffling the cards if they aren't on-board ones? --=20 Eygene _ ___ _.--. # \`.|\..----...-'` `-._.-'_.-'` # Remember that it is hard / ' ` , __.--' # to read the on-line manual =20 )/' _/ \ `-_, / # while single-stepping the kernel. `-'" `"\_ ,_.-;_.-\_ ', fsc/as # _.-'_./ {_.' ; / # -- FreeBSD Developers handbook=20 {_.-``-' {_/ # --E/DnYTRukya0zdZ1 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (FreeBSD) iEYEARECAAYFAkkcAfkACgkQthUKNsbL7Yj1QACdGEPqxsAuofV1xCAHEXu/7Mpe +uwAmgPBJB25aYHyETiZFg9ghQYLajfi =B8HH -----END PGP SIGNATURE----- --E/DnYTRukya0zdZ1-- From owner-freebsd-hardware@FreeBSD.ORG Thu Nov 13 11:09:32 2008 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EDD9D1065689; Thu, 13 Nov 2008 11:09:32 +0000 (UTC) (envelope-from fbsd@dannysplace.net) Received: from mail.dannysplace.net (mail.dannysplace.net [213.133.54.210]) by mx1.freebsd.org (Postfix) with ESMTP id A2C308FC0A; Thu, 13 Nov 2008 11:09:32 +0000 (UTC) (envelope-from fbsd@dannysplace.net) Received: from 203-206-171-212.perm.iinet.net.au ([203.206.171.212] helo=[192.168.10.10]) by mail.dannysplace.net with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.69 (FreeBSD)) (envelope-from ) id 1L0a4o-000KWE-0I; Thu, 13 Nov 2008 21:09:31 +1000 Message-ID: <491C0B00.4030408@dannysplace.net> Date: Thu, 13 Nov 2008 21:09:52 +1000 From: Danny Carroll User-Agent: Thunderbird 2.0.0.17 (Windows/20080914) MIME-Version: 1.0 To: Willem Jan Withagen References: <490A782F.9060406@dannysplace.net> <20081031033208.GA21220@icarus.home.lan> <490A849C.7030009@dannysplace.net> <20081031043412.GA22289@icarus.home.lan> <490A8FAD.8060009@dannysplace.net> <491BBF38.9010908@dannysplace.net> <491BE632.1020801@IMAP> In-Reply-To: <491BE632.1020801@IMAP> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Authenticated-User: danny X-Authenticator: plain X-Sender-Verify: SUCCEEDED (sender exists & accepts mail) X-Exim-Version: 4.69 (build at 08-Jul-2008 08:59:40) X-Date: 2008-11-13 21:09:30 X-Connected-IP: 203.206.171.212:3604 X-Message-Linecount: 55 X-Body-Linecount: 40 X-Message-Size: 2035 X-Body-Size: 1246 X-Received-Count: 1 X-Recipient-Count: 4 X-Local-Recipient-Count: 4 X-Local-Recipient-Defer-Count: 0 X-Local-Recipient-Fail-Count: 0 X-SA-Exim-Connect-IP: 203.206.171.212 X-SA-Exim-Rcpt-To: wjw@digiware.nl, freebsd-fs@freebsd.org, koitsu@FreeBSD.org, freebsd-hardware@freebsd.org X-SA-Exim-Mail-From: fbsd@dannysplace.net X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on ferrari.dannysplace.net X-Spam-Level: X-Spam-Status: No, score=0.2 required=8.0 tests=ALL_TRUSTED,TVD_RCVD_IP autolearn=disabled version=3.2.5 X-SA-Exim-Version: 4.2 X-SA-Exim-Scanned: Yes (on mail.dannysplace.net) Cc: freebsd-fs@freebsd.org, Jeremy Chadwick , freebsd-hardware@freebsd.org Subject: Re: Areca vs. ZFS performance testing. X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: fbsd@dannysplace.net List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Nov 2008 11:09:33 -0000 Good idea. Actually, what I will do eventually is *also* post the results to the mailing list. It will probably be around long after my own server is gone. -D Willem Jan Withagen wrote: > Danny Carroll wrote: >> Danny Carroll wrote: >>> Jeremy Chadwick wrote: >>>> I'd like to see the performance difference between these scenarios: >>>> >>>> - Memory cache enabled on Areca, write caching enabled on disks >>>> - Memory cache enabled on Areca, write caching disabled on disks >>>> - Memory cache disabled on Areca, write caching enabled on disks >>>> - Memory cache disabled on Areca, write caching disabled on disks >>>> >> >> >> The initial results for a ICH9 vs Areca in JBod mode can be found here: >> http://www.dannysplace.net/ZFS-JBODTests.html > > Just as a polite question, since I'm very much in favor doing > benchmarking and do appreciate these kinds of test. > > You might want to add an introductory page to your results describing > how you setup the test: > Details of the hardware > Details of the disk setup > possible version and options with bonnie > The script you used.... > > This would allow others to redo your experiment and try to figure out > why their numbers are different. > > --WjW > > From owner-freebsd-hardware@FreeBSD.ORG Thu Nov 13 13:58:57 2008 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0B5441065686; Thu, 13 Nov 2008 13:58:57 +0000 (UTC) (envelope-from danny@dannysplace.net) Received: from mail.dannysplace.net (mail.dannysplace.net [213.133.54.210]) by mx1.freebsd.org (Postfix) with ESMTP id AE8278FC12; Thu, 13 Nov 2008 13:58:56 +0000 (UTC) (envelope-from danny@dannysplace.net) Received: from 203-206-171-212.perm.iinet.net.au ([203.206.171.212] helo=[192.168.10.10]) by mail.dannysplace.net with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.69 (FreeBSD)) (envelope-from ) id 1L0cij-000LW7-6j; Thu, 13 Nov 2008 23:58:55 +1000 Message-ID: <491C32BF.7020805@dannysplace.net> Date: Thu, 13 Nov 2008 23:59:27 +1000 From: Danny Carroll User-Agent: Thunderbird 2.0.0.17 (Windows/20080914) MIME-Version: 1.0 To: Jeremy Chadwick References: <491BBF38.9010908@dannysplace.net> <200811130657.GAA26763@sopwith.solgatos.com> <20081113074301.GA13938@icarus.home.lan> In-Reply-To: <20081113074301.GA13938@icarus.home.lan> X-Enigmail-Version: 0.95.7 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Authenticated-User: danny X-Authenticator: plain X-Sender-Verify: SUCCEEDED (sender exists & accepts mail) X-Exim-Version: 4.69 (build at 08-Jul-2008 08:59:40) X-Date: 2008-11-13 23:58:53 X-Connected-IP: 203.206.171.212:3668 X-Message-Linecount: 85 X-Body-Linecount: 70 X-Message-Size: 3740 X-Body-Size: 3062 X-Received-Count: 1 X-Recipient-Count: 4 X-Local-Recipient-Count: 4 X-Local-Recipient-Defer-Count: 0 X-Local-Recipient-Fail-Count: 0 X-SA-Exim-Connect-IP: 203.206.171.212 X-SA-Exim-Rcpt-To: koitsu@FreeBSD.org, freebsd@sopwith.solgatos.com, freebsd-fs@freebsd.org, freebsd-hardware@freebsd.org X-SA-Exim-Mail-From: danny@dannysplace.net X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on ferrari.dannysplace.net X-Spam-Level: X-Spam-Status: No, score=0.2 required=8.0 tests=ALL_TRUSTED,AWL,TVD_RCVD_IP autolearn=disabled version=3.2.5 X-SA-Exim-Version: 4.2 X-SA-Exim-Scanned: Yes (on mail.dannysplace.net) Cc: freebsd-fs@freebsd.org, Dieter , freebsd-hardware@freebsd.org Subject: Re: Areca vs. ZFS performance testing. X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Nov 2008 13:58:57 -0000 Jeremy Chadwick wrote: > On Wed, Nov 12, 2008 at 10:57:58PM +0000, Dieter wrote: > The Areca controller he has can do caching of its own (it has 256MBytes > of cache). Meaning, if you disable write cache on the disks (but not > the Areca controller itself), all of the caching being done is purely > controller-based. The actual disk writes between the controller and the > disk will, of course, be "slow" -- but between the OS and the > controller, things should appear fast. It is entirely possible. I do not know however if the Areca cache works just for Raid or also in JBOD mode. The card can be configured via a web interface (it has it's own nic), via the CLI, or via the BIOS. The only setting I do see is: Disk Write Cache Mode. This is what I have tested. It might have been the Areca cache I turned off, or it might have been the disk caches that I turned off. I hope it is the former, otherwise what is the purpose of having a battery backup unit? If the disks cache the write, then you will probably lose data anyway. I think, once I turn on Raid mode, there will be an option to turn on/off caching in the raid part of the config. The manual shows me that there is an option there, but it only indicates that you can change the cache mode from WriteBack to WriteThrough. But for now, since it's in JBOD mode I cannot access that. > Let me outline the 4 test scenarios (I thought I did this in my original > mail to Danny, but I believe I also said "don't get caught up in > excessive granularity because it'll just confuse people now" -- case in > point): > > - Areca cache disabled, disk write cache enabled > - Areca cache disabled, disk write cache disabled > - Areca cache enabled, disk write cache enabled > - Areca cache enabled, disk write cache disabled [**] > > As I understand it, Danny performed the tests with the [**] > configuration. > The tests should have names: Test 1: Areca cache disabled, disk write cache enabled Test 2: Areca cache disabled, disk write cache disabled Test 3: Areca cache enabled, disk write cache enabled Test 4: Areca cache enabled, disk write cache disabled You did outline these, I thought I was performaing test 2 because I am assuming that when you turn on JBOD mode, you do not get caching on the controller. Once I am sure there is not something glaringly wrong with the FreeBSD side of things I'll run as many of these tests as I can. For now, I think it is only tests 1 and 2. So, my thoughts remain, why was the read performance the same, and the write performance actually marginally better, after I turned off the cache? I did a reboot after I turned off the cache but I did not power cycle the drives. Perhaps that is the answer? Or perhaps simply the Areca controller cannot turn off the cache on the ST31000340AS drives. Or perhaps the cache is ALWAYS enabled and cannot be turned off on the controller. That mean I was doing test 4 as Jeremy suggested. That seems a likely possibility as well. In fact, thinking about it now, it makes the most sense to me. -D From owner-freebsd-hardware@FreeBSD.ORG Thu Nov 13 15:29:31 2008 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 01C24106564A for ; Thu, 13 Nov 2008 15:29:31 +0000 (UTC) (envelope-from ndenev@gmail.com) Received: from yx-out-2324.google.com (yx-out-2324.google.com [74.125.44.29]) by mx1.freebsd.org (Postfix) with ESMTP id AA08C8FC1F for ; Thu, 13 Nov 2008 15:29:30 +0000 (UTC) (envelope-from ndenev@gmail.com) Received: by yx-out-2324.google.com with SMTP id 8so410634yxb.13 for ; Thu, 13 Nov 2008 07:29:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:cc:message-id:from:to :in-reply-to:content-type:content-transfer-encoding:mime-version :subject:date:references:x-pgp-agent:x-mailer; bh=ctg0OpkmrRsFeUfU/6qX5CbQMTncGNmDRUb4qc+CRBM=; b=qvmqylTjwbKsb0GPQ2QpHlliSOGEoGMOj5eopmB9eOlp5jkmmd393o3kMG1QEHibQd cD8dAQUXvnq55mhYfuX1VB4sWu2eeHKT22+3EFGHbLAUyCTevwMinH8Dn6HUdcJH+v12 0rhPmrbDMgnoBhvWrCplXdpBaOsKkQA9r0ItI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=cc:message-id:from:to:in-reply-to:content-type :content-transfer-encoding:mime-version:subject:date:references :x-pgp-agent:x-mailer; b=l/O87vKCoCl0e6ahZZFS9+nNbhwB67oMLv89KtXdnCbGe9X6lYXiUTSSwDvXug+QeP 2VqwVWawYsIb6zVYqKtxFgcTgLODzkDBhl1Xy6gp+CiVTX/YFXADCRT/5/Oh5NdFkxPj zUuunDcMW5WZBeokqmncjcr27eP3zm9GVlR0c= Received: by 10.102.228.10 with SMTP id a10mr5855328muh.26.1226588768996; Thu, 13 Nov 2008 07:06:08 -0800 (PST) Received: from ndenev.cmotd.com (blah.sun-fish.com [217.18.249.150]) by mx.google.com with ESMTPS id w5sm47456622mue.10.2008.11.13.07.06.05 (version=TLSv1/SSLv3 cipher=RC4-MD5); Thu, 13 Nov 2008 07:06:06 -0800 (PST) Message-Id: <04B6F041-3052-4650-BE62-817E2B28D034@gmail.com> From: Nikolay Denev To: Danny Carroll In-Reply-To: <491C32BF.7020805@dannysplace.net> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v929.2) Date: Thu, 13 Nov 2008 17:06:03 +0200 References: <491BBF38.9010908@dannysplace.net> <200811130657.GAA26763@sopwith.solgatos.com> <20081113074301.GA13938@icarus.home.lan> <491C32BF.7020805@dannysplace.net> X-Pgp-Agent: GPGMail d53 (v53, Leopard) X-Mailer: Apple Mail (2.929.2) Cc: freebsd-fs@freebsd.org, Jeremy Chadwick , Dieter , freebsd-hardware@freebsd.org Subject: Re: Areca vs. ZFS performance testing. X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Nov 2008 15:29:31 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 13 Nov, 2008, at 15:59 , Danny Carroll wrote: [snip] > > It is entirely possible. I do not know however if the Areca cache > works > just for Raid or also in JBOD mode. > I think some RAID controllers do not use the cache when you export the disks as pass-thru/jbod, but on some controllers you can workaround this by making every disk a RAID0(stripe) array with only one disk. Dunno if that would work on the areca... [snip] - -- Regards, Nikolay Denev -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (Darwin) iEYEARECAAYFAkkcQlsACgkQHNAJ/fLbfrkTkgCgo2NupY2Qe3TglJpoIIwne4uH VRwAnRl9p44NFxyWf9zhjrZOOImtiBAs =4Djt -----END PGP SIGNATURE----- From owner-freebsd-hardware@FreeBSD.ORG Thu Nov 13 17:15:19 2008 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 441AF10656A9; Thu, 13 Nov 2008 17:15:19 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id D862A8FC16; Thu, 13 Nov 2008 17:15:18 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from phobos.local ([192.168.254.200]) (authenticated bits=0) by pooker.samsco.org (8.14.2/8.14.2) with ESMTP id mADGnhcE040068; Thu, 13 Nov 2008 09:49:44 -0700 (MST) (envelope-from scottl@samsco.org) Message-ID: <491C5AA7.1030004@samsco.org> Date: Thu, 13 Nov 2008 09:49:43 -0700 From: Scott Long User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.13) Gecko/20080313 SeaMonkey/1.1.9 MIME-Version: 1.0 To: Danny Carroll References: <490A782F.9060406@dannysplace.net> <20081031033208.GA21220@icarus.home.lan> <490A849C.7030009@dannysplace.net> <20081031043412.GA22289@icarus.home.lan> <490A8FAD.8060009@dannysplace.net> <491BBF38.9010908@dannysplace.net> In-Reply-To: <491BBF38.9010908@dannysplace.net> X-Enigmail-Version: 0.95.6 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-4.4 required=3.8 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.1.8 X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on pooker.samsco.org Cc: freebsd-fs@freebsd.org, Jeremy Chadwick , fbsd@dannysplace.net, freebsd-hardware@freebsd.org Subject: Re: Areca vs. ZFS performance testing. X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Nov 2008 17:15:19 -0000 Danny Carroll wrote: > Danny Carroll wrote: >> Jeremy Chadwick wrote: >>> I'd like to see the performance difference between these scenarios: >>> >>> - Memory cache enabled on Areca, write caching enabled on disks >>> - Memory cache enabled on Areca, write caching disabled on disks >>> - Memory cache disabled on Areca, write caching enabled on disks >>> - Memory cache disabled on Areca, write caching disabled on disks >>> > > > The initial results for a ICH9 vs Areca in JBod mode can be found here: > http://www.dannysplace.net/ZFS-JBODTests.html > > Summary: > 5 Disk ZFS RaidZ array with atime turned off. > ICH9 - block reads avg 400MByte/Sec > ICH9 - block writes avg 150MByte/Sec > ArecaJBOD - block reads avg 300MByte/Sec > ArecaJBOD - block writes avg 160MByte/Sec > > > The Areca seems to be in all except char and block writes. Block reads > are 75% as fast as the ICH9 and rewrites are about 85% as fast. > > There seems to be little difference between enabling and disabling the > disk cache on the Areca. This leads me to two conclusions: > 1. Disabling the write cache does nothing on Seagate drives. > 2. IO to the drives is so slow that a write cache is irrelevant. > > These are just some quick tests that I started with, mainly to compare > the areca bus versus the ich9 bus. If someone has any tuning > suggestions, then now is the time to make them before I migrate the ICH9 > drives to the Areca bus. The Areca controller likely doesn't buffer/cache for disks in JBOD mode, as others in this thread have stated. Without buffering, simple disk controllers will almost always be faster than accelerated raid controllers because the accelerated controllers add more latency between the host and the disk. A simple controller will directly funnel data from the host to the disk as soon as it receives a command. An accelerated controller, however, has a CPU and a mini-OS on it that has to schedule the work coming from the host and handle its own tasks and interrupts. This adds latency that quickly adds up under benchmarks. Your numbers clearly demonstrate this. Scott From owner-freebsd-hardware@FreeBSD.ORG Thu Nov 13 19:46:55 2008 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A9BC01065677 for ; Thu, 13 Nov 2008 19:46:55 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from server.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id 302488FC08 for ; Thu, 13 Nov 2008 19:46:55 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from localhost.corp.yahoo.com (john@localhost [IPv6:::1]) (authenticated bits=0) by server.baldwin.cx (8.14.3/8.14.3) with ESMTP id mADJkO2q096236; Thu, 13 Nov 2008 14:46:49 -0500 (EST) (envelope-from jhb@freebsd.org) From: John Baldwin To: freebsd-hardware@freebsd.org Date: Thu, 13 Nov 2008 14:36:57 -0500 User-Agent: KMail/1.9.7 References: <20475350.post@talk.nabble.com> In-Reply-To: <20475350.post@talk.nabble.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200811131436.57422.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (server.baldwin.cx [IPv6:::1]); Thu, 13 Nov 2008 14:46:49 -0500 (EST) X-Virus-Scanned: ClamAV 0.93.1/8628/Thu Nov 13 10:57:02 2008 on server.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.6 required=4.2 tests=AWL,BAYES_00,NO_RELAYS, UPPERCASE_25_50 autolearn=ham version=3.1.3 X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on server.baldwin.cx Cc: "weinter.lim" Subject: Re: Problems With BroadCom NetXtreme Ethernet and Atheros AR5B91 X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Nov 2008 19:46:55 -0000 On Thursday 13 November 2008 01:18:03 am weinter.lim wrote: > > I am using FreeBSD 7.1 BETA 2 on my New Acer Aspire 4530 > During Boot Dmesg did not detect my Ethernet BroadCom NetXtreme > pciconf -lv > > none5@pci0:8:0:0 Class=0X020000 card=0X014a1025 chip=0X168414e4 rev=0X10 > hdr=0X00 > vendor = 'Broadcom Corporation' > class = network > subclass = ethernet The Linux tg3 driver claims this is a BCM5764. Can you try this patch and include any messages (especially any phy messages) from a verbose boot? Alternatively, you could boot w/o bge in the kernel, turn on bootverbose (debug.bootverbose sysctl) and kldload a patched bge.ko and capture the dmesg output. Index: if_bge.c =================================================================== RCS file: /usr/cvs/src/sys/dev/bge/if_bge.c,v retrieving revision 1.215 diff -u -r1.215 if_bge.c --- if_bge.c 27 Oct 2008 22:10:01 -0000 1.215 +++ if_bge.c 13 Nov 2008 19:33:15 -0000 @@ -184,6 +184,7 @@ { BCOM_VENDORID, BCOM_DEVICEID_BCM5754M }, { BCOM_VENDORID, BCOM_DEVICEID_BCM5755 }, { BCOM_VENDORID, BCOM_DEVICEID_BCM5755M }, + { BCOM_VENDORID, BCOM_DEVICEID_BCM5764 }, { BCOM_VENDORID, BCOM_DEVICEID_BCM5780 }, { BCOM_VENDORID, BCOM_DEVICEID_BCM5780S }, { BCOM_VENDORID, BCOM_DEVICEID_BCM5781 }, Index: if_bgereg.h =================================================================== RCS file: /usr/cvs/src/sys/dev/bge/if_bgereg.h,v retrieving revision 1.81 diff -u -r1.81 if_bgereg.h --- if_bgereg.h 14 Oct 2008 20:28:42 -0000 1.81 +++ if_bgereg.h 13 Nov 2008 19:33:02 -0000 @@ -2094,6 +2094,7 @@ #define BCOM_DEVICEID_BCM5754M 0x1672 #define BCOM_DEVICEID_BCM5755 0x167B #define BCOM_DEVICEID_BCM5755M 0x1673 +#define BCOM_DEVICEID_BCM5764 0x1684 #define BCOM_DEVICEID_BCM5780 0x166A #define BCOM_DEVICEID_BCM5780S 0x166B #define BCOM_DEVICEID_BCM5781 0x16DD > > My Atheros Wireless also failed to detect > > none6@pci0:11:0:0 Class=0X028000 card=0X03031a32 chip=0X002a168c rev=0X01 > hdr=0X00 > vendor = 'Atheros Communication Inc' > class = network No idea on this one. You could ask Sam perhaps. -- John Baldwin From owner-freebsd-hardware@FreeBSD.ORG Thu Nov 13 19:47:01 2008 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3F5571065677; Thu, 13 Nov 2008 19:47:01 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from server.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id D47808FC1B; Thu, 13 Nov 2008 19:47:00 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from localhost.corp.yahoo.com (john@localhost [IPv6:::1]) (authenticated bits=0) by server.baldwin.cx (8.14.3/8.14.3) with ESMTP id mADJkO2r096236; Thu, 13 Nov 2008 14:46:54 -0500 (EST) (envelope-from jhb@freebsd.org) From: John Baldwin To: freebsd-hardware@freebsd.org Date: Thu, 13 Nov 2008 14:38:25 -0500 User-Agent: KMail/1.9.7 References: <704830.24415.qm@web45815.mail.sp1.yahoo.com> <366483.43588.qm@web45807.mail.sp1.yahoo.com> <20081113081936.GA14779@icarus.home.lan> In-Reply-To: <20081113081936.GA14779@icarus.home.lan> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200811131438.25904.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (server.baldwin.cx [IPv6:::1]); Thu, 13 Nov 2008 14:46:55 -0500 (EST) X-Virus-Scanned: ClamAV 0.93.1/8628/Thu Nov 13 10:57:02 2008 on server.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.6 required=4.2 tests=AWL,BAYES_00,NO_RELAYS autolearn=ham version=3.1.3 X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on server.baldwin.cx Cc: Jeremy Chadwick , Won De Erick Subject: Re: IRQ31 and IRQ32 on HPDL585 running FreeBSD 7.0 are consuming HIGH CPU usage X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Nov 2008 19:47:01 -0000 On Thursday 13 November 2008 03:19:36 am Jeremy Chadwick wrote: > On Thu, Nov 13, 2008 at 12:07:37AM -0800, Won De Erick wrote: > > Noted on this, I will update you through this thread. > > > > However is there any possibility of the following: > > > > > I don't know if there's a way to split the interrupt request for each bce's Rx and Tx, > > > which means a total of four IRQs, and eventually four cores (or 4 CPUs) > > > for the transactions. With this way, the IDLE processors would be utilized. > > > > What I mean here is, for the two interfaces: > > > > one IRQ for bce0 Rx > > one IRQ for bce0 Tx > > one IRQ for bce1 Rx > > one IRQ for bce1 Tx > > I can't even begin to imagine how this would be possible on any NIC. igb(4) does it. It is quite possible and one of the purposes of MSI. However, the current bce(4) hardware does not support this. It only allows for a single message and thus a single IRQ per-device. -- John Baldwin From owner-freebsd-hardware@FreeBSD.ORG Thu Nov 13 19:47:07 2008 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 353EA1065677; Thu, 13 Nov 2008 19:47:07 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from server.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id B29948FC17; Thu, 13 Nov 2008 19:47:06 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from localhost.corp.yahoo.com (john@localhost [IPv6:::1]) (authenticated bits=0) by server.baldwin.cx (8.14.3/8.14.3) with ESMTP id mADJkO2s096236; Thu, 13 Nov 2008 14:47:00 -0500 (EST) (envelope-from jhb@freebsd.org) From: John Baldwin To: freebsd-hardware@freebsd.org Date: Thu, 13 Nov 2008 14:41:16 -0500 User-Agent: KMail/1.9.7 References: <491BFB68.7050405@infoweapons.com> In-Reply-To: <491BFB68.7050405@infoweapons.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200811131441.16427.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (server.baldwin.cx [IPv6:::1]); Thu, 13 Nov 2008 14:47:00 -0500 (EST) X-Virus-Scanned: ClamAV 0.93.1/8628/Thu Nov 13 10:57:02 2008 on server.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.6 required=4.2 tests=AWL,BAYES_00,NO_RELAYS autolearn=ham version=3.1.3 X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on server.baldwin.cx Cc: freebsd-hackers@freebsd.org, "Ronnel P. Maglasang" Subject: Re: assigning interrupts X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Nov 2008 19:47:07 -0000 On Thursday 13 November 2008 05:03:20 am Ronnel P. Maglasang wrote: > Hi All, > > Is there a way to explicitly assign an interrupt > of a device? I'm running on 6.3 and the two NICs > share the same interrupt. Obviously this will affect > the performance if the NICs are exposed to heavy network > traffic. > > # vmstat -i > interrupt total rate > > irq11: em0 vr0+ 1081099 77 > > Total 16958562 1222 > > > Looking at the driver's code, I have the initial though > that this is the place where I can modify. > > -- > adapter->res_interrupt = bus_alloc_resource_any(dev, > SYS_RES_IRQ, &rid, RF_SHAREABLE | RF_ACTIVE); > -- > > I've tried changing RF_SHAREABLE to RF_ALLOCATED or other > values but still could not get the desired result and worst > the device fail to initialize. Is this possible in 6.3? You can not easily assign them, no. In many cases the interrupt pins from the devices may be hardwired to a single input pin on an interrupt controller. In that case there is nothing you can do. You can read more about the gory details here: http://people.freebsd.org/~jhb/papers/bsdcan/2007/ -- John Baldwin From owner-freebsd-hardware@FreeBSD.ORG Thu Nov 13 20:46:47 2008 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0613B10656A5; Thu, 13 Nov 2008 20:46:47 +0000 (UTC) (envelope-from fbsd@dannysplace.net) Received: from mail.dannysplace.net (mail.dannysplace.net [213.133.54.210]) by mx1.freebsd.org (Postfix) with ESMTP id AB6A38FC22; Thu, 13 Nov 2008 20:46:46 +0000 (UTC) (envelope-from fbsd@dannysplace.net) Received: from 203-206-171-212.perm.iinet.net.au ([203.206.171.212] helo=[192.168.10.10]) by mail.dannysplace.net with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.69 (FreeBSD)) (envelope-from ) id 1L0j5P-000L3h-Fk; Fri, 14 Nov 2008 06:46:45 +1000 Message-ID: <491C9224.4050407@dannysplace.net> Date: Fri, 14 Nov 2008 06:46:28 +1000 From: Danny Carroll User-Agent: Thunderbird 2.0.0.17 (Windows/20080914) MIME-Version: 1.0 To: Nikolay Denev References: <491BBF38.9010908@dannysplace.net> <200811130657.GAA26763@sopwith.solgatos.com> <20081113074301.GA13938@icarus.home.lan> <491C32BF.7020805@dannysplace.net> <04B6F041-3052-4650-BE62-817E2B28D034@gmail.com> In-Reply-To: <04B6F041-3052-4650-BE62-817E2B28D034@gmail.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Authenticated-User: danny X-Authenticator: plain X-Sender-Verify: SUCCEEDED (sender exists & accepts mail) X-Exim-Version: 4.69 (build at 08-Jul-2008 08:59:40) X-Date: 2008-11-14 06:46:43 X-Connected-IP: 203.206.171.212:1163 X-Message-Linecount: 34 X-Body-Linecount: 18 X-Message-Size: 1416 X-Body-Size: 608 X-Received-Count: 1 X-Recipient-Count: 5 X-Local-Recipient-Count: 5 X-Local-Recipient-Defer-Count: 0 X-Local-Recipient-Fail-Count: 0 X-SA-Exim-Connect-IP: 203.206.171.212 X-SA-Exim-Rcpt-To: ndenev@gmail.com, koitsu@FreeBSD.org, freebsd-fs@freebsd.org, freebsd@sopwith.solgatos.com, freebsd-hardware@freebsd.org X-SA-Exim-Mail-From: fbsd@dannysplace.net X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on ferrari.dannysplace.net X-Spam-Level: X-Spam-Status: No, score=0.2 required=8.0 tests=ALL_TRUSTED,AWL,TVD_RCVD_IP autolearn=disabled version=3.2.5 X-SA-Exim-Version: 4.2 X-SA-Exim-Scanned: Yes (on mail.dannysplace.net) Cc: freebsd-fs@freebsd.org, Jeremy Chadwick , Dieter , freebsd-hardware@freebsd.org Subject: Re: Areca vs. ZFS performance testing. X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: fbsd@dannysplace.net List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Nov 2008 20:46:47 -0000 Nikolay Denev wrote: > I think some RAID controllers do not use the cache when you export the > disks as pass-thru/jbod, I assumed this is the case but now I am not so sure. > but on some controllers you can workaround this by making > every disk a RAID0(stripe) array with only one disk. > Dunno if that would work on the areca... You can probably do that with this as controller as well. However if I look at the manual I do not see an option to disable the cache for Raid sets. Only to change it from Write-back to Write-Through. I guess write-through is *almost* as if the cache is disabled. -D From owner-freebsd-hardware@FreeBSD.ORG Thu Nov 13 20:59:43 2008 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7B3F51065674; Thu, 13 Nov 2008 20:59:43 +0000 (UTC) (envelope-from fbsd@dannysplace.net) Received: from mail.dannysplace.net (mail.dannysplace.net [213.133.54.210]) by mx1.freebsd.org (Postfix) with ESMTP id 2BF858FC0A; Thu, 13 Nov 2008 20:59:43 +0000 (UTC) (envelope-from fbsd@dannysplace.net) Received: from 203-206-171-212.perm.iinet.net.au ([203.206.171.212] helo=[192.168.10.10]) by mail.dannysplace.net with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.69 (FreeBSD)) (envelope-from ) id 1L0jHv-000L89-BJ; Fri, 14 Nov 2008 06:59:41 +1000 Message-ID: <491C9535.3030504@dannysplace.net> Date: Fri, 14 Nov 2008 06:59:33 +1000 From: Danny Carroll User-Agent: Thunderbird 2.0.0.17 (Windows/20080914) MIME-Version: 1.0 To: Scott Long References: <490A782F.9060406@dannysplace.net> <20081031033208.GA21220@icarus.home.lan> <490A849C.7030009@dannysplace.net> <20081031043412.GA22289@icarus.home.lan> <490A8FAD.8060009@dannysplace.net> <491BBF38.9010908@dannysplace.net> <491C5AA7.1030004@samsco.org> In-Reply-To: <491C5AA7.1030004@samsco.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Authenticated-User: danny X-Authenticator: plain X-Sender-Verify: SUCCEEDED (sender exists & accepts mail) X-Exim-Version: 4.69 (build at 08-Jul-2008 08:59:40) X-Date: 2008-11-14 06:59:39 X-Connected-IP: 203.206.171.212:1171 X-Message-Linecount: 44 X-Body-Linecount: 29 X-Message-Size: 2326 X-Body-Size: 1532 X-Received-Count: 1 X-Recipient-Count: 4 X-Local-Recipient-Count: 4 X-Local-Recipient-Defer-Count: 0 X-Local-Recipient-Fail-Count: 0 X-SA-Exim-Connect-IP: 203.206.171.212 X-SA-Exim-Rcpt-To: scottl@samsco.org, freebsd-fs@freebsd.org, koitsu@freebsd.org, freebsd-hardware@freebsd.org X-SA-Exim-Mail-From: fbsd@dannysplace.net X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on ferrari.dannysplace.net X-Spam-Level: X-Spam-Status: No, score=0.2 required=8.0 tests=ALL_TRUSTED,TVD_RCVD_IP autolearn=disabled version=3.2.5 X-SA-Exim-Version: 4.2 X-SA-Exim-Scanned: Yes (on mail.dannysplace.net) Cc: freebsd-fs@freebsd.org, Jeremy Chadwick , freebsd-hardware@freebsd.org Subject: Re: Areca vs. ZFS performance testing. X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: fbsd@dannysplace.net List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Nov 2008 20:59:43 -0000 Scott Long wrote: > The Areca controller likely doesn't buffer/cache for disks in JBOD mode, > as others in this thread have stated. Without buffering, simple disk > controllers will almost always be faster than accelerated raid > controllers because the accelerated controllers add more latency between > the host and the disk. A simple controller will directly funnel data > from the host to the disk as soon as it receives a command. An > accelerated controller, however, has a CPU and a mini-OS on it that has > to schedule the work coming from the host and handle its own tasks and > interrupts. This adds latency that quickly adds up under benchmarks. > Your numbers clearly demonstrate this. That's nice to know. I'm not sure it tells us why the Non-Cached writes were about 8% faster though. The other thing about the "NoWriteCache" test I performed that I neglected to mention yesterday is that I actually panic'd the box (running out of memory). This was the first time I have had that happen with ZFS even though in previous testing (with cache enabled) I punished the box for a lot longer. Perhaps the ZFS caching took over where the disk caching left off? Could that explain why I did not see a negative difference in the numbers between Cache enabled and Cache disabled? One of the questions I wanted to answer for myself was just this: "Does a battery-backed cache on an Areca card protect me when I am in JBOD mode." If the Areca does not buffer/cache in JBOD mode then that means the answer is no. -D From owner-freebsd-hardware@FreeBSD.ORG Fri Nov 14 06:17:50 2008 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 792521065677 for ; Fri, 14 Nov 2008 06:17:50 +0000 (UTC) (envelope-from bounces@nabble.com) Received: from kuber.nabble.com (kuber.nabble.com [216.139.236.158]) by mx1.freebsd.org (Postfix) with ESMTP id 293E18FC13 for ; Fri, 14 Nov 2008 06:17:49 +0000 (UTC) (envelope-from bounces@nabble.com) Received: from isper.nabble.com ([192.168.236.156]) by kuber.nabble.com with esmtp (Exim 4.63) (envelope-from ) id 1L0s04-0007nZ-FX for freebsd-hardware@freebsd.org; Thu, 13 Nov 2008 22:17:48 -0800 Message-ID: <20495276.post@talk.nabble.com> Date: Thu, 13 Nov 2008 22:17:48 -0800 (PST) From: "weinter.lim" To: freebsd-hardware@freebsd.org In-Reply-To: <200811131436.57422.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Nabble-From: limguowei@gmail.com References: <20475350.post@talk.nabble.com> <200811131436.57422.jhb@freebsd.org> Subject: Re: Problems With BroadCom NetXtreme Ethernet and Atheros AR5B91 X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Nov 2008 06:17:50 -0000 John Baldwin wrote: > > On Thursday 13 November 2008 01:18:03 am weinter.lim wrote: >> >> I am using FreeBSD 7.1 BETA 2 on my New Acer Aspire 4530 >> During Boot Dmesg did not detect my Ethernet BroadCom NetXtreme >> pciconf -lv >> >> none5@pci0:8:0:0 Class=0X020000 card=0X014a1025 chip=0X168414e4 rev=0X10 >> hdr=0X00 >> vendor = 'Broadcom Corporation' >> class = network >> subclass = ethernet > > The Linux tg3 driver claims this is a BCM5764. Can you try this patch and > include any messages (especially any phy messages) from a verbose boot? > Alternatively, you could boot w/o bge in the kernel, turn on bootverbose > (debug.bootverbose sysctl) and kldload a patched bge.ko and capture the > dmesg > output. > > Index: if_bge.c > =================================================================== > RCS file: /usr/cvs/src/sys/dev/bge/if_bge.c,v > retrieving revision 1.215 > diff -u -r1.215 if_bge.c > --- if_bge.c 27 Oct 2008 22:10:01 -0000 1.215 > +++ if_bge.c 13 Nov 2008 19:33:15 -0000 > @@ -184,6 +184,7 @@ > { BCOM_VENDORID, BCOM_DEVICEID_BCM5754M }, > { BCOM_VENDORID, BCOM_DEVICEID_BCM5755 }, > { BCOM_VENDORID, BCOM_DEVICEID_BCM5755M }, > + { BCOM_VENDORID, BCOM_DEVICEID_BCM5764 }, > { BCOM_VENDORID, BCOM_DEVICEID_BCM5780 }, > { BCOM_VENDORID, BCOM_DEVICEID_BCM5780S }, > { BCOM_VENDORID, BCOM_DEVICEID_BCM5781 }, > Index: if_bgereg.h > =================================================================== > RCS file: /usr/cvs/src/sys/dev/bge/if_bgereg.h,v > retrieving revision 1.81 > diff -u -r1.81 if_bgereg.h > --- if_bgereg.h 14 Oct 2008 20:28:42 -0000 1.81 > +++ if_bgereg.h 13 Nov 2008 19:33:02 -0000 > @@ -2094,6 +2094,7 @@ > #define BCOM_DEVICEID_BCM5754M 0x1672 > #define BCOM_DEVICEID_BCM5755 0x167B > #define BCOM_DEVICEID_BCM5755M 0x1673 > +#define BCOM_DEVICEID_BCM5764 0x1684 > #define BCOM_DEVICEID_BCM5780 0x166A > #define BCOM_DEVICEID_BCM5780S 0x166B > #define BCOM_DEVICEID_BCM5781 0x16DD > >> >> My Atheros Wireless also failed to detect >> >> none6@pci0:11:0:0 Class=0X028000 card=0X03031a32 chip=0X002a168c rev=0X01 >> hdr=0X00 >> vendor = 'Atheros Communication Inc' >> class = network > > No idea on this one. You could ask Sam perhaps. > > -- > John Baldwin > _______________________________________________ > freebsd-hardware@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hardware > To unsubscribe, send any mail to > "freebsd-hardware-unsubscribe@freebsd.org" > > Hi I just found another issue that is rather worrying When i tried compiling my kernel the temperature run up to 90 degs On my previous laptop Acer Aspire 4520G using the TL-60 Processor it went up to a maximium of 70 deg Now on my new RM-72 Turion It went up to 90 deg I am afraid that even if the heat didn't burn up my processor it will burn up my other components as well8-O I am really worried -- View this message in context: http://www.nabble.com/Problems-With-BroadCom-NetXtreme-Ethernet-and-Atheros-AR5B91-tp20475350p20495276.html Sent from the freebsd-hardware mailing list archive at Nabble.com. From owner-freebsd-hardware@FreeBSD.ORG Fri Nov 14 08:26:46 2008 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8A4B61065672; Fri, 14 Nov 2008 08:26:46 +0000 (UTC) (envelope-from rmaglasang@infoweapons.com) Received: from infoweapons.com (mail0.infoweapons.net [204.2.248.50]) by mx1.freebsd.org (Postfix) with ESMTP id 2A95A8FC18; Fri, 14 Nov 2008 08:26:45 +0000 (UTC) (envelope-from rmaglasang@infoweapons.com) Received: from ([58.71.34.146]) by mail0.infoweapons.com with ESMTP id 4321444.1331974; Fri, 14 Nov 2008 03:26:19 -0500 Received: from [10.3.1.41] ([10.3.1.41]) by cebexch01.cebu.infoweapons.com over TLS secured channel with Microsoft SMTPSVC(6.0.3790.3959); Fri, 14 Nov 2008 16:26:19 +0800 Message-ID: <491D356A.70607@infoweapons.com> Date: Fri, 14 Nov 2008 16:23:06 +0800 From: "Ronnel P. Maglasang" User-Agent: Thunderbird 1.5 (X11/20060613) MIME-Version: 1.0 To: John Baldwin References: <491BFB68.7050405@infoweapons.com> <200811131441.16427.jhb@freebsd.org> In-Reply-To: <200811131441.16427.jhb@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 14 Nov 2008 08:26:19.0417 (UTC) FILETIME=[AB5E6890:01C94632] Cc: freebsd-hackers@freebsd.org, freebsd-hardware@freebsd.org Subject: Re: assigning interrupts X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Nov 2008 08:26:46 -0000 John Baldwin wrote: > On Thursday 13 November 2008 05:03:20 am Ronnel P. Maglasang wrote: > >> Hi All, >> >> Is there a way to explicitly assign an interrupt >> of a device? I'm running on 6.3 and the two NICs >> share the same interrupt. Obviously this will affect >> the performance if the NICs are exposed to heavy network >> traffic. >> >> # vmstat -i >> interrupt total rate >> >> irq11: em0 vr0+ 1081099 77 >> >> Total 16958562 1222 >> >> >> Looking at the driver's code, I have the initial though >> that this is the place where I can modify. >> >> -- >> adapter->res_interrupt = bus_alloc_resource_any(dev, >> SYS_RES_IRQ, &rid, RF_SHAREABLE | RF_ACTIVE); >> -- >> >> I've tried changing RF_SHAREABLE to RF_ALLOCATED or other >> values but still could not get the desired result and worst >> the device fail to initialize. Is this possible in 6.3? >> > > You can not easily assign them, no. In many cases the interrupt pins from the > devices may be hardwired to a single input pin on an interrupt controller. > In that case there is nothing you can do. You can read more about the gory > details here: > > http://people.freebsd.org/~jhb/papers/bsdcan/2007/ > > What was changed in 7.x in terms of assigning interrupts? I have another box running on 7.0 (2 NICs). I noticed there are no devices sharing interrupts. But if 6.x is installed on the same box (previous installation), the two NICs will share the same interrupt. I'm now looking at the drivers. I assume this is not NIC-firmware related. From owner-freebsd-hardware@FreeBSD.ORG Fri Nov 14 08:48:50 2008 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4C405106567A for ; Fri, 14 Nov 2008 08:48:50 +0000 (UTC) (envelope-from won.derick@yahoo.com) Received: from n19.bullet.mail.mud.yahoo.com (n19.bullet.mail.mud.yahoo.com [68.142.206.146]) by mx1.freebsd.org (Postfix) with SMTP id E9B7B8FC13 for ; Fri, 14 Nov 2008 08:48:49 +0000 (UTC) (envelope-from won.derick@yahoo.com) Received: from [68.142.200.227] by n19.bullet.mail.mud.yahoo.com with NNFMP; 14 Nov 2008 08:48:49 -0000 Received: from [68.142.201.248] by t8.bullet.mud.yahoo.com with NNFMP; 14 Nov 2008 08:48:49 -0000 Received: from [127.0.0.1] by omp409.mail.mud.yahoo.com with NNFMP; 14 Nov 2008 08:48:49 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 353751.97999.bm@omp409.mail.mud.yahoo.com Received: (qmail 39969 invoked by uid 60001); 14 Nov 2008 08:48:48 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Mailer:References:Date:From:Subject:To:Cc:MIME-Version:Content-Type:Message-ID; b=cYkj4V8Bnnv7t+qoR8O6bvZfSKJwuvPUxl636t5B8bre9V+crO2VC2zXTI2A4462JflqTXzkeQDTQm4dA/cZHwruCmcfEa8MzBGsCr2uFjtDruBiat7qgPDmlFS8L3GEB8daik9PZEU7klkEoaAd6bvi7kWNzkpP6naiLi/TRgM=; X-YMail-OSG: MvMKZjYVM1lQTrMbazYVvTvAnX4hXMuly9kT539ALTPBaHNEsqgP0L0Zxo0Kl.GSJ.xiuno0.t90oPpPvyPOF0IkzOwbFNjUe_QnSJr7H4KU0C_ca2Y8wzaC1E74jei1tpBcIy1RHXE_hy9MjBw6JNlwB054ofrTtxGQnK85spj3TsNhyZexMeO4uv1A Received: from [58.71.34.137] by web45811.mail.sp1.yahoo.com via HTTP; Fri, 14 Nov 2008 00:48:48 PST X-Mailer: YahooMailRC/1155.20 YahooMailWebService/0.7.260.1 References: <704830.24415.qm@web45815.mail.sp1.yahoo.com> <366483.43588.qm@web45807.mail.sp1.yahoo.com> <20081113081936.GA14779@icarus.home.lan> <200811131438.25904.jhb@freebsd.org> Date: Fri, 14 Nov 2008 00:48:48 -0800 (PST) From: Won De Erick To: John Baldwin , freebsd-hardware@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Message-ID: <829178.39837.qm@web45811.mail.sp1.yahoo.com> Cc: Jeremy Chadwick Subject: Re: IRQ31 and IRQ32 on HPDL585 running FreeBSD 7.0 are consuming HIGH CPU usage X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Nov 2008 08:48:50 -0000 > ----- Original Message ---- > From: John Baldwin > To: freebsd-hardware@freebsd.org > Cc: Jeremy Chadwick ; Won De Erick > Sent: Friday, November 14, 2008 3:38:25 AM > Subject: Re: IRQ31 and IRQ32 on HPDL585 running FreeBSD 7.0 are consuming HIGH CPU usage > > On Thursday 13 November 2008 03:19:36 am Jeremy Chadwick wrote: > > On Thu, Nov 13, 2008 at 12:07:37AM -0800, Won De Erick wrote: > > > Noted on this, I will update you through this thread. > > > > > > However is there any possibility of the following: > > > > > > > I don't know if there's a way to split the interrupt request for each bce's Rx and Tx, > > > > which means a total of four IRQs, and eventually four cores (or 4 CPUs) > > > > for the transactions. With this way, the IDLE processors would be utilized. > > > > > > What I mean here is, for the two interfaces: > > > > > > one IRQ for bce0 Rx > > > one IRQ for bce0 Tx > > > one IRQ for bce1 Rx > > > one IRQ for bce1 Tx > > > > I can't even begin to imagine how this would be possible on any NIC. > igb(4) does it. It is quite possible and one of the purposes of MSI. > However, the current bce(4) hardware does not support this. It only allows > for a single message and thus a single IRQ per-device. based from the man pages, igb driver supports Intel NICs w/ controllers starting from Intel NIC controller 82574. One Intel NIC (controller: 82576, see http://www.intel.com/Assets/PDF/prodbrief/320116.pdf) says it supports MSIX which minimizes the overhead of interrupts and allows load balancing of interrupt handling between multiple cores/CPUs. I should want a little more explanation how this feature being handled by MSIX. Thanks a lot. > -- > John Baldwin From owner-freebsd-hardware@FreeBSD.ORG Fri Nov 14 11:30:11 2008 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 571891065670 for ; Fri, 14 Nov 2008 11:30:11 +0000 (UTC) (envelope-from won.derick@yahoo.com) Received: from n29.bullet.mail.mud.yahoo.com (n29.bullet.mail.mud.yahoo.com [68.142.207.48]) by mx1.freebsd.org (Postfix) with SMTP id 2889D8FC14 for ; Fri, 14 Nov 2008 11:30:11 +0000 (UTC) (envelope-from won.derick@yahoo.com) Received: from [68.142.200.221] by n29.bullet.mail.mud.yahoo.com with NNFMP; 14 Nov 2008 11:30:10 -0000 Received: from [68.142.201.242] by t9.bullet.mud.yahoo.com with NNFMP; 14 Nov 2008 11:30:10 -0000 Received: from [127.0.0.1] by omp403.mail.mud.yahoo.com with NNFMP; 14 Nov 2008 11:30:10 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 774753.17095.bm@omp403.mail.mud.yahoo.com Received: (qmail 77148 invoked by uid 60001); 14 Nov 2008 11:30:10 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:MIME-Version:Content-Type:Message-ID; b=eEq/RjgdiNSv3WojhDEJfT44+SJjHfM0Tw9mP+9huVEdJLskq1UfUbv3u1Sw7ILl7B1F9UhGR7i53vBSvi00XmJixZ6fsuUIewXEGWlJhJMsyPDSx/oZqPWo94IxPlCGjVa4KObI0nmvm0cdU161gkLahWqovJ2nZd/jd4j8HzI=; X-YMail-OSG: pz_Ei3kVM1nqJS_7_q7b9SlPAnBH51L5eHd5kISihmCJ9g3DFMlnS7wlpPMhkDKpZE2hG8Xvgy2aO178VCIGc.2uqdrPzQVRfYHynhMJTgzkRiDULEZjmL5uKnwbH8a.ePvkBr3ykuCmp7EVdNR40eqAPBs- Received: from [58.71.34.137] by web45809.mail.sp1.yahoo.com via HTTP; Fri, 14 Nov 2008 03:30:10 PST X-Mailer: YahooMailRC/1155.20 YahooMailWebService/0.7.260.1 Date: Fri, 14 Nov 2008 03:30:10 -0800 (PST) From: Won De Erick To: Jeremy Chadwick MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Message-ID: <305614.76266.qm@web45809.mail.sp1.yahoo.com> Cc: freebsd-net@freebsd.org, freebsd-hardware@freebsd.org Subject: Re: IRQ31 and IRQ32 on HPDL585 running FreeBSD 7.0 are consuming HIGH CPU usage X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Nov 2008 11:30:11 -0000 > ----- Original Message ---- > From: Won De Erick > To: Jeremy Chadwick > Cc: freebsd-hardware@freebsd.org > Sent: Thursday, November 13, 2008 4:07:37 PM > Subject: Re: IRQ31 and IRQ32 on HPDL585 running FreeBSD 7.0 are consuming HIGH CPU usage > > Noted on this, I will update you through this thread. With FreeBSD 7.1 Beta2, here is the result: 52 root 1 -68 - 0K 16K CPU11 b 38:43 95.36% irq32: bce1 51 root 1 -68 - 0K 16K CPU10 a 25:50 85.16% irq31: bce0 There's a slight difference w/ the previous result though, but I observed that overall CPU utilization didn't change. ### Complete result: PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 21 root 1 171 ki31 0K 16K CPU5 5 91:29 100.00% idle: cpu5 14 root 1 171 ki31 0K 16K CPU12 c 91:29 100.00% idle: cpu12 22 root 1 171 ki31 0K 16K CPU4 4 91:29 100.00% idle: cpu4 17 root 1 171 ki31 0K 16K CPU9 9 91:28 100.00% idle: cpu9 12 root 1 171 ki31 0K 16K CPU14 e 91:27 100.00% idle: cpu14 23 root 1 171 ki31 0K 16K CPU3 3 90:58 100.00% idle: cpu3 25 root 1 171 ki31 0K 16K CPU1 1 86:41 100.00% idle: cpu1 19 root 1 171 ki31 0K 16K CPU7 7 84:00 100.00% idle: cpu7 24 root 1 171 ki31 0K 16K CPU2 2 83:53 100.00% idle: cpu2 18 root 1 171 ki31 0K 16K CPU8 8 79:01 100.00% idle: cpu8 26 root 1 171 ki31 0K 16K RUN 0 74:18 100.00% idle: cpu0 11 root 1 171 ki31 0K 16K CPU15 0 0:00 100.00% idle: cpu15 13 root 1 171 ki31 0K 16K CPU13 0 0:00 100.00% idle: cpu13 20 root 1 171 ki31 0K 16K CPU6 6 90:18 99.27% idle: cpu6 52 root 1 -68 - 0K 16K CPU11 b 38:43 95.36% irq32: bce1 51 root 1 -68 - 0K 16K CPU10 a 25:50 85.16% irq31: bce0 16 root 1 171 ki31 0K 16K RUN a 65:39 15.97% idle: cpu10 28 root 1 -32 - 0K 16K WAIT 8 12:28 5.18% swi4: clock sio 15 root 1 171 ki31 0K 16K RUN b 52:46 3.76% idle: cpu11 45 root 1 -64 - 0K 16K WAIT 7 7:29 1.17% irq17: uhci0 47 root 1 -64 - 0K 16K WAIT 6 1:11 0.10% irq16: ciss0 Is there some ways how the intensive [network] load can be distributed among the IDLE processors? Another thing, I observed that in the above test, the net.isr is enabled by default. When I tried disabling this, # sysctl net.isr.direct=0 net.isr.direct: 1 -> 0 the result: 52 root 1 -68 - 0K 16K WAIT b 64:00 42.97% irq32: bce1 51 root 1 -68 - 0K 16K WAIT a 38:22 12.26% irq31: bce0 The CPU utilizations considerably dropped! What was changed in the fbsd7.1? How useful this net.isr is? Is this needed in an environment with heavy network traffic? Can someone explain? > However is there any possibility of the following: > > > I don't know if there's a way to split the interrupt request for each bce's Rx and Tx, > > which means a total of four IRQs, and eventually four cores (or 4 CPUs) > > for the transactions. With this way, the IDLE processors would be utilized. > > What I mean here is, for the two interfaces: > > one IRQ for bce0 Rx > one IRQ for bce0 Tx > one IRQ for bce1 Rx > one IRQ for bce1 Tx > > > Thanks, > > Won > > > > ________________________________ > From: Jeremy Chadwick > To: Won De Erick > Cc: freebsd-hardware@freebsd.org > Sent: Thursday, November 13, 2008 3:46:02 PM > Subject: Re: IRQ31 and IRQ32 on HPDL585 running FreeBSD 7.0 are consuming HIGH CPU usage > > On Wed, Nov 12, 2008 at 11:38:15PM -0800, Won De Erick wrote: > > I am conducting a CPU utilization testing with my box(HP DL 585 running FreeBSD 7.0), and come up with the results below: > > > > 52 root 1 -68 - 0K 16K CPU11 b 123:53 100.00% irq32: bce1 > > 51 root 1 -68 - 0K 16K CPU10 a 119:28 89.06% irq31: bce0 > > > > irq31 and irq32 are consuming high CPU usage, which i think the cause of hard reset. > > There was a ***major*** bce(4) cleanup that just happened. Your 7.0 box > will not have these changes. Please upgrade your box to RELENG_7 > (a.k.a. 7.1-PRERELEASE), csup'd recently (today preferably), and try > your tests again: > > http://lists.freebsd.org/pipermail/freebsd-stable/2008-November/046482.html > > -- > | Jeremy Chadwick jdc at parodius.com | > | Parodius Networking http://www.parodius.com/ | > | UNIX Systems Administrator Mountain View, CA, USA | > | Making life hard for others since 1977. PGP: 4BD6C0CB | > From owner-freebsd-hardware@FreeBSD.ORG Fri Nov 14 11:48:58 2008 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 33508106567D for ; Fri, 14 Nov 2008 11:48:58 +0000 (UTC) (envelope-from freebsd-hardware@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id E2B458FC1A for ; Fri, 14 Nov 2008 11:48:57 +0000 (UTC) (envelope-from freebsd-hardware@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1L0xAT-0002fH-7Y for freebsd-hardware@freebsd.org; Fri, 14 Nov 2008 11:48:53 +0000 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 14 Nov 2008 11:48:53 +0000 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 14 Nov 2008 11:48:53 +0000 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-hardware@freebsd.org From: Ivan Voras Date: Fri, 14 Nov 2008 12:49:13 +0100 Lines: 43 Message-ID: References: <305614.76266.qm@web45809.mail.sp1.yahoo.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig52BA60C1C46196FA69E6A61E" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Thunderbird 2.0.0.17 (X11/20080925) In-Reply-To: <305614.76266.qm@web45809.mail.sp1.yahoo.com> X-Enigmail-Version: 0.95.0 Sender: news Cc: freebsd-net@freebsd.org Subject: Re: IRQ31 and IRQ32 on HPDL585 running FreeBSD 7.0 are consuming HIGH CPU usage X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Nov 2008 11:48:58 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig52BA60C1C46196FA69E6A61E Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Won De Erick wrote: > Another thing, I observed that in the above test, the net.isr is enable= d by default. When I tried disabling this, >=20 > # sysctl net.isr.direct=3D0 > net.isr.direct: 1 -> 0 >=20 > the result: >=20 > 52 root 1 -68 - 0K 16K WAIT b 64:00 42.97% irq32: b= ce1 > 51 root 1 -68 - 0K 16K WAIT a 38:22 12.26% irq31: b= ce0 >=20 > The CPU utilizations considerably dropped! You will probably find a "swi" process that has picked up the difference (when isr.direct is disabled, some of network protocol processing is offloaded to a swi thread). This might help spread the load across CPU but in my testing it didn't help real-world throughput. --------------enig52BA60C1C46196FA69E6A61E Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJHWW5ldnAQVacBcgRAuSIAKCTKl2OQhfLfdZEko958W+sLSOk8QCfTGwe NuARD4/ZI0qppdyEmyfMtYU= =1918 -----END PGP SIGNATURE----- --------------enig52BA60C1C46196FA69E6A61E-- From owner-freebsd-hardware@FreeBSD.ORG Fri Nov 14 12:46:40 2008 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1D95110656E1 for ; Fri, 14 Nov 2008 12:46:40 +0000 (UTC) (envelope-from won.derick@yahoo.com) Received: from n61.bullet.mail.sp1.yahoo.com (n61.bullet.mail.sp1.yahoo.com [98.136.44.37]) by mx1.freebsd.org (Postfix) with SMTP id 055678FC1B for ; Fri, 14 Nov 2008 12:46:40 +0000 (UTC) (envelope-from won.derick@yahoo.com) Received: from [69.147.65.150] by n61.bullet.mail.sp1.yahoo.com with NNFMP; 14 Nov 2008 12:46:39 -0000 Received: from [69.147.65.152] by t7.bullet.mail.sp1.yahoo.com with NNFMP; 14 Nov 2008 12:46:39 -0000 Received: from [127.0.0.1] by omp400.mail.sp1.yahoo.com with NNFMP; 14 Nov 2008 12:46:39 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 664345.46462.bm@omp400.mail.sp1.yahoo.com Received: (qmail 71360 invoked by uid 60001); 14 Nov 2008 12:46:39 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Mailer:References:Date:From:Subject:To:Cc:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-ID; b=6SZdsnWb9tYh24v7/F/vKd/tkRDS+iD50DG27BzgJsbrBwSRfveJ9iEvarhFuMFGU9B7MKuJKZ13neXgowUWD7hCOfbMErVBEdcebB5+y1LUQvbZ/A5WM4rwwA02fJU8Ni3kY5NC7y+Ymrjz95GfQZwaxuxN7wRObvb4/xFp2yI=; X-YMail-OSG: 9fVYlgQVM1kft5b2vXyxbzjOsOWq_9W92k.C_b5_3wwls5U3y2K73ig5HyxWPpSeUbKZf9w.bZZAp4U8dGqi7g54olZ8NigT1ijtmqSVCDooE6o65AMg29bghtUCFWzUkG89DI7TKIlr4V_4xwTVcdg0eBc- Received: from [58.71.34.137] by web45806.mail.sp1.yahoo.com via HTTP; Fri, 14 Nov 2008 04:46:39 PST X-Mailer: YahooMailRC/1155.20 YahooMailWebService/0.7.260.1 References: <305614.76266.qm@web45809.mail.sp1.yahoo.com> Date: Fri, 14 Nov 2008 04:46:39 -0800 (PST) From: Won De Erick To: Ivan Voras , freebsd-hardware@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Message-ID: <555333.76711.qm@web45806.mail.sp1.yahoo.com> Cc: freebsd-net@freebsd.org Subject: Re: IRQ31 and IRQ32 on HPDL585 running FreeBSD 7.0 are consuming HIGH CPU usage X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Nov 2008 12:46:40 -0000 > ----- Original Message ----=0A=0A> From: Ivan Voras = =0A> To: freebsd-hardware@freebsd.org=0A> Cc: freebsd-net@freebsd.org=0A> S= ent: Friday, November 14, 2008 7:49:13 PM=0A> Subject: Re: IRQ31 and IRQ32 = on HPDL585 running FreeBSD 7.0 are consuming HIGH CPU usage=0A> =0A> Won De= Erick wrote:=0A> =0A> > Another thing, I observed that in the above test, = the net.isr is enabled by default. When I tried disabling this,=0A> > =0A> = > # sysctl net.isr.direct=3D0=0A> > net.isr.direct: 1 -> 0=0A> > =0A> > the= result:=0A> > =0A> > 52 root 1 -68 - 0K 16K WAIT b 64:= 00 42.97% irq32: bce1=0A> > 51 root 1 -68 - 0K 16K WAIT = a 38:22 12.26% irq31: bce0=0A> > =0A> > The CPU utilizations considerably = dropped!=0A> =0A> You will probably find a "swi" process that has picked up= the difference=0A> (when isr.direct is disabled, some of network protocol = processing is=0A> offloaded to a swi thread). This might help spread the lo= ad across CPU=0A> but in my testing it didn't help real-world throughput.= =0A>=0A=0AYou are right. I noticed the following when net.isr is diabled, l= owering the idle time of cpu0.=0A=0A 27 root 1 -44 - 0K = 16K WAIT 0 52:20 76.37% swi1: net=0A 26 root 1 171 ki31 0K = 16K CPU0 0 111:58 64.36% idle: cpu0=0A=0A=0AAnother thing,=0APacket dr= ops on Intel NIC ( Intel=AE PRO/1000 PT Dual Port Server Adapter w/ control= processor 82571GB) did not occur when the net.isr was disabled, while the = overall CPU utilization remains considerably low.=0A=0ANote: The following = result was obtained during a transition from a disabled to enabled net.isr.= Hence the first part=0A=0Apackets errs bytes packets errs b= ytes colls drops=0A 10844 0 15603850 7940 0 582934 = 0 0=0A 11659 0 16800328 8503 0 630330 0= 0=0A 11778 0 17033560 8998 0 677934 0 = 0=0A 12149 0 17592134 9504 0 728094 0 0=0A = 12551 0 18223550 9974 0 774164 0 0=0A 1= 3127 0 19093604 10413 0 811858 0 0=0A 13712 = 0 20010140 10924 0 848014 0 0=0A 14499 0= 21153538 11407 0 878252 0 0=0A 14818 0 21= 740270 11979 0 915374 0 0=0A 15831 0 2313644= 6 12376 0 950636 0 0=0A 15912 0 23365454 = 12852 0 997242 0 0=0A 16257 0 23848866 132= 82 0 1041878 0 0=0A 16384 0 24084782 13666 = 0 1079790 0 0=0A 16670 0 24508980 14078 0 = 1106886 0 0=0A 17845 0 26255548 14486 0 113= 4700 0 0=0A 18097 0 26705634 15064 0 1163308 = 0 0=0A 18470 0 27283000 15365 0 1198828 0= 0=0A 18139 0 26842676 15596 0 1225540 0 = 0=0A 18792 0 27799564 16000 0 1264568 0 0=0A = 17854 178 26454106 16521 0 1298714 0 0=0A 1= 6741 1542 24820298 16770 0 1343328 0 0=0A = input (em0) output=0A packets errs bytes pac= kets errs bytes colls drops=0A 15288 1667 22683486 17231 = 0 1422690 0 0=0A 15539 1718 23250372 17282 0= 1495058 0 0=0A 14379 545 21541954 17364 0 1= 508696 0 0=0A 14312 1733 21546776 17276 0 150337= 2 0 0=0A 14269 1744 21498908 17516 0 1508294 = 0 0=0A 14444 1729 21766812 17175 0 1482130 0 = 0=0A 15023 1724 22643198 16987 0 1432048 0 0= =0A 15279 1703 23036294 16909 0 1395094 0 0=0A = 15325 1701 23118536 16938 0 1380268 0 0=0A 15= 572 1684 23494362 16909 0 1344214 0 0=0A 15798 = 1699 23845972 16857 0 1303200 0 0=0A 16195 1683 = 24497790 17059 0 1291586 0 0=0A 16431 1674 248= 51278 16826 0 1245320 0 0=0A 16683 1643 25231910= 16675 0 1204450 0 0=0A 16728 1647 25302534 = 16672 0 1178930 0 0=0A 16826 1649 25455662 1666= 2 0 1178140 0 0=0A 16760 1653 25352830 16480 = 0 1161086 0 0=0A 17002 1634 25720672 16423 0 = 1143508 0 0=0A 16943 1643 25629892 16642 0 1160= 858 0 0=0A 16995 1644 25708823 16539 0 1153782 = 0 0=0A 17026 1643 25758462 16606 0 1153342 0 = 0=0A=0AHowever, network throughput didn't change in the two scenarios a= bove.=0AIs there anything that I can test to improve my network throughput.= =0A=0AThanks,=0A=0AWon=0A=0A=0A=0A From owner-freebsd-hardware@FreeBSD.ORG Fri Nov 14 12:54:52 2008 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2DA92106564A for ; Fri, 14 Nov 2008 12:54:52 +0000 (UTC) (envelope-from freebsd-hardware@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id 6A57C8FC08 for ; Fri, 14 Nov 2008 12:54:51 +0000 (UTC) (envelope-from freebsd-hardware@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1L0yCG-00055f-94 for freebsd-hardware@freebsd.org; Fri, 14 Nov 2008 12:54:48 +0000 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 14 Nov 2008 12:54:48 +0000 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 14 Nov 2008 12:54:48 +0000 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-hardware@freebsd.org From: Ivan Voras Date: Fri, 14 Nov 2008 13:55:13 +0100 Lines: 41 Message-ID: References: <305614.76266.qm@web45809.mail.sp1.yahoo.com> <555333.76711.qm@web45806.mail.sp1.yahoo.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigBEDEAAAD4EFBF68438E74D08" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Thunderbird 2.0.0.17 (X11/20080925) In-Reply-To: <555333.76711.qm@web45806.mail.sp1.yahoo.com> X-Enigmail-Version: 0.95.0 Sender: news Cc: freebsd-net@freebsd.org Subject: Re: IRQ31 and IRQ32 on HPDL585 running FreeBSD 7.0 are consuming HIGH CPU usage X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Nov 2008 12:54:52 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigBEDEAAAD4EFBF68438E74D08 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Won De Erick wrote: > 17002 1634 25720672 16423 0 1143508 0 0 > 16943 1643 25629892 16642 0 1160858 0 0 > 16995 1644 25708823 16539 0 1153782 0 0 > 17026 1643 25758462 16606 0 1153342 0 0 What do you use for generating the traffic? > However, network throughput didn't change in the two scenarios above. > Is there anything that I can test to improve my network throughput. I'm not sure but you could maybe try disabling all but one of your CPUs and see if anything changes. It looks like you have at least 16 CPUs. On my machine, with 8 CPUs (2xquad core) I get roughly 150,000 PPS before I hit a similar single-CPU-bound network processing bottleneck (also with Intel=C2=AE PRO/1000 PT/Server) - which is too low for my needs so I'm searching for a solution. In my case I don't see transmission errors. --------------enigBEDEAAAD4EFBF68438E74D08 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJHXUyldnAQVacBcgRAhSxAKCCRAh6L0XvOFJzmfP9cjd5NIQc6QCcCtJF 9PauafcND6O9BSDmm/D3dLE= =Nz/N -----END PGP SIGNATURE----- --------------enigBEDEAAAD4EFBF68438E74D08-- From owner-freebsd-hardware@FreeBSD.ORG Fri Nov 14 13:14:14 2008 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2F0D61065686 for ; Fri, 14 Nov 2008 13:14:14 +0000 (UTC) (envelope-from won.derick@yahoo.com) Received: from n57.bullet.mail.sp1.yahoo.com (n57.bullet.mail.sp1.yahoo.com [98.136.44.49]) by mx1.freebsd.org (Postfix) with SMTP id 155778FC1A for ; Fri, 14 Nov 2008 13:14:13 +0000 (UTC) (envelope-from won.derick@yahoo.com) Received: from [69.147.65.172] by n57.bullet.mail.sp1.yahoo.com with NNFMP; 14 Nov 2008 13:14:13 -0000 Received: from [69.147.84.116] by t14.bullet.mail.sp1.yahoo.com with NNFMP; 14 Nov 2008 13:14:13 -0000 Received: from [127.0.0.1] by omp208.mail.sp1.yahoo.com with NNFMP; 14 Nov 2008 13:14:13 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 842488.33215.bm@omp208.mail.sp1.yahoo.com Received: (qmail 7387 invoked by uid 60001); 14 Nov 2008 13:14:13 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Mailer:References:Date:From:Subject:To:Cc:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-ID; b=bFhWQ2GM3juB3+Jus3z0xAjeneHicjhzNi9YY5Mx5oRdGqepMTudFkB5c3mGhCjw/1dwlmb7rTnABcZK76gZx3Oh+znAJjBtRqKQR5VqpbWOIp3s13w5Xx8ofdoSYodoMyuGQyU74CfwvPpmvSZCLqGBJhiMROuEhFZQ15UUcUM=; X-YMail-OSG: ONhFMVkVM1naSNJhlt2e6USwxXd.S8GC0Ju.F6gok7TloSoqrUYZe.6_MS6Gi9.wSKg2IAteFYjIcK1N1veTPYutw8n44gz1IrWHmPTbPjJM.hR9Bl7CisKPmwrYMlGSrTDfnDI2Wr1bmY9kBIz8cwjQGoM- Received: from [58.71.34.137] by web45801.mail.sp1.yahoo.com via HTTP; Fri, 14 Nov 2008 05:14:13 PST X-Mailer: YahooMailRC/1155.20 YahooMailWebService/0.7.260.1 References: <305614.76266.qm@web45809.mail.sp1.yahoo.com> <555333.76711.qm@web45806.mail.sp1.yahoo.com> Date: Fri, 14 Nov 2008 05:14:13 -0800 (PST) From: Won De Erick To: Ivan Voras , freebsd-hardware@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Message-ID: <674901.2796.qm@web45801.mail.sp1.yahoo.com> X-Mailman-Approved-At: Fri, 14 Nov 2008 14:10:25 +0000 Cc: freebsd-net@freebsd.org Subject: Re: IRQ31 and IRQ32 on HPDL585 running FreeBSD 7.0 are consuming HIGH CPU usage X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Nov 2008 13:14:14 -0000 > From: Ivan Voras =0A=0A> To: freebsd-hardware@freebsd= ..org=0A> Cc: freebsd-net@freebsd.org=0A> Sent: Friday, November 14, 2008 8:= 55:13 PM=0A> Subject: Re: IRQ31 and IRQ32 on HPDL585 running FreeBSD 7.0 ar= e consuming HIGH CPU usage=0A> =0A> Won De Erick wrote:=0A> =0A> > 170= 02 1634 25720672 16423 0 1143508 0 0=0A> > 1694= 3 1643 25629892 16642 0 1160858 0 0=0A> > 16995= 1644 25708823 16539 0 1153782 0 0=0A> > 17026 = 1643 25758462 16606 0 1153342 0 0=0A> =0A> What do y= ou use for generating the traffic?=0A> =0A=0AI used w-get in bombarding tcp= traffic w/ thousand of sessions.=0A=0A> > However, network throughput didn= 't change in the two scenarios above.=0A> > Is there anything that I can te= st to improve my network throughput.=0A> =0A> I'm not sure but you could ma= ybe try disabling all but one of your CPUs=0A> and see if anything changes.= It looks like you have at least 16 CPUs. On=0A> my machine, with 8 CPUs (2= xquad core) I get roughly 150,000 PPS before I=0A> hit a similar single-CPU= -bound network processing bottleneck (also with=0A> Intel=AE PRO/1000 PT/Se= rver) - which is too low for my needs so I'm=0A> searching for a solution. = In my case I don't see transmission errors.=0A=0Awill try this.=0A=0A=0A=0A= From owner-freebsd-hardware@FreeBSD.ORG Fri Nov 14 16:13:49 2008 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 924D91065676 for ; Fri, 14 Nov 2008 16:13:49 +0000 (UTC) (envelope-from bounces@nabble.com) Received: from kuber.nabble.com (kuber.nabble.com [216.139.236.158]) by mx1.freebsd.org (Postfix) with ESMTP id 692BF8FC14 for ; Fri, 14 Nov 2008 16:13:49 +0000 (UTC) (envelope-from bounces@nabble.com) Received: from isper.nabble.com ([192.168.236.156]) by kuber.nabble.com with esmtp (Exim 4.63) (envelope-from ) id 1L11Iq-0005n0-M2 for freebsd-hardware@freebsd.org; Fri, 14 Nov 2008 08:13:48 -0800 Message-ID: <20503489.post@talk.nabble.com> Date: Fri, 14 Nov 2008 08:13:48 -0800 (PST) From: "weinter.lim" To: freebsd-hardware@freebsd.org In-Reply-To: <200811131436.57422.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Nabble-From: limguowei@gmail.com References: <20475350.post@talk.nabble.com> <200811131436.57422.jhb@freebsd.org> Subject: Re: Problems With BroadCom NetXtreme Ethernet and Atheros AR5B91 X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Nov 2008 16:13:49 -0000 On Thursday 13 November 2008 01:18:03 am weinter.lim wrote: > > I am using FreeBSD 7.1 BETA 2 on my New Acer Aspire 4530 > During Boot Dmesg did not detect my Ethernet BroadCom NetXtreme > pciconf -lv > > none5@pci0:8:0:0 Class=0X020000 card=0X014a1025 chip=0X168414e4 rev=0X10 > hdr=0X00 > vendor = 'Broadcom Corporation' > class = network > subclass = ethernet The Linux tg3 driver claims this is a BCM5764. Can you try this patch and include any messages (especially any phy messages) from a verbose boot? Alternatively, you could boot w/o bge in the kernel, turn on bootverbose (debug.bootverbose sysctl) and kldload a patched bge.ko and capture the dmesg output. Index: if_bge.c =================================================================== RCS file: /usr/cvs/src/sys/dev/bge/if_bge.c,v retrieving revision 1.215 diff -u -r1.215 if_bge.c --- if_bge.c 27 Oct 2008 22:10:01 -0000 1.215 +++ if_bge.c 13 Nov 2008 19:33:15 -0000 @@ -184,6 +184,7 @@ { BCOM_VENDORID, BCOM_DEVICEID_BCM5754M }, { BCOM_VENDORID, BCOM_DEVICEID_BCM5755 }, { BCOM_VENDORID, BCOM_DEVICEID_BCM5755M }, + { BCOM_VENDORID, BCOM_DEVICEID_BCM5764 }, { BCOM_VENDORID, BCOM_DEVICEID_BCM5780 }, { BCOM_VENDORID, BCOM_DEVICEID_BCM5780S }, { BCOM_VENDORID, BCOM_DEVICEID_BCM5781 }, Index: if_bgereg.h =================================================================== RCS file: /usr/cvs/src/sys/dev/bge/if_bgereg.h,v retrieving revision 1.81 diff -u -r1.81 if_bgereg.h --- if_bgereg.h 14 Oct 2008 20:28:42 -0000 1.81 +++ if_bgereg.h 13 Nov 2008 19:33:02 -0000 @@ -2094,6 +2094,7 @@ #define BCOM_DEVICEID_BCM5754M 0x1672 #define BCOM_DEVICEID_BCM5755 0x167B #define BCOM_DEVICEID_BCM5755M 0x1673 +#define BCOM_DEVICEID_BCM5764 0x1684 #define BCOM_DEVICEID_BCM5780 0x166A #define BCOM_DEVICEID_BCM5780S 0x166B #define BCOM_DEVICEID_BCM5781 0x16DD Hi, I patched the files as advise the Ethernet was detected but upon configuring the network no data was transmitted (the activity light did not lit up) and the system hung Thanks -- View this message in context: http://www.nabble.com/Problems-With-BroadCom-NetXtreme-Ethernet-and-Atheros-AR5B91-tp20475350p20503489.html Sent from the freebsd-hardware mailing list archive at Nabble.com. From owner-freebsd-hardware@FreeBSD.ORG Sat Nov 15 18:01:37 2008 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C16DD1065670; Sat, 15 Nov 2008 18:01:37 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from server.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id 10AF98FC12; Sat, 15 Nov 2008 18:01:36 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from localhost.corp.yahoo.com (john@localhost [IPv6:::1]) (authenticated bits=0) by server.baldwin.cx (8.14.3/8.14.3) with ESMTP id mAFI1NmS015493; Sat, 15 Nov 2008 13:01:30 -0500 (EST) (envelope-from jhb@freebsd.org) From: John Baldwin To: "Ronnel P. Maglasang" Date: Sat, 15 Nov 2008 11:23:40 -0500 User-Agent: KMail/1.9.7 References: <491BFB68.7050405@infoweapons.com> <200811131441.16427.jhb@freebsd.org> <491D356A.70607@infoweapons.com> In-Reply-To: <491D356A.70607@infoweapons.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200811151123.41183.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (server.baldwin.cx [IPv6:::1]); Sat, 15 Nov 2008 13:01:30 -0500 (EST) X-Virus-Scanned: ClamAV 0.93.1/8636/Sat Nov 15 00:05:47 2008 on server.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.6 required=4.2 tests=AWL,BAYES_00,NO_RELAYS autolearn=ham version=3.1.3 X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on server.baldwin.cx Cc: freebsd-hackers@freebsd.org, freebsd-hardware@freebsd.org Subject: Re: assigning interrupts X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Nov 2008 18:01:37 -0000 On Friday 14 November 2008 03:23:06 am Ronnel P. Maglasang wrote: > John Baldwin wrote: > > On Thursday 13 November 2008 05:03:20 am Ronnel P. Maglasang wrote: > > > >> Hi All, > >> > >> Is there a way to explicitly assign an interrupt > >> of a device? I'm running on 6.3 and the two NICs > >> share the same interrupt. Obviously this will affect > >> the performance if the NICs are exposed to heavy network > >> traffic. > >> > >> # vmstat -i > >> interrupt total rate > >> > >> irq11: em0 vr0+ 1081099 77 > >> > >> Total 16958562 1222 > >> > >> > >> Looking at the driver's code, I have the initial though > >> that this is the place where I can modify. > >> > >> -- > >> adapter->res_interrupt = bus_alloc_resource_any(dev, > >> SYS_RES_IRQ, &rid, RF_SHAREABLE | RF_ACTIVE); > >> -- > >> > >> I've tried changing RF_SHAREABLE to RF_ALLOCATED or other > >> values but still could not get the desired result and worst > >> the device fail to initialize. Is this possible in 6.3? > >> > > > > You can not easily assign them, no. In many cases the interrupt pins from the > > devices may be hardwired to a single input pin on an interrupt controller. > > In that case there is nothing you can do. You can read more about the gory > > details here: > > > > http://people.freebsd.org/~jhb/papers/bsdcan/2007/ > > > > > What was changed in 7.x in terms of assigning interrupts? I have another > box running on 7.0 (2 NICs). I noticed there are no devices sharing > interrupts. But if 6.x is installed on the same box (previous installation), > the two NICs will share the same interrupt. I'm now looking at the drivers. > I assume this is not NIC-firmware related. MSI. Newer 6.x (6.4, possibly 6.3; if not by default on 6.3 then it can be enabled on 6.3 via 'hw.pci.enable_msi=1' tunable) will do it as well. -- John Baldwin From owner-freebsd-hardware@FreeBSD.ORG Sat Nov 15 18:01:45 2008 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D8AFD106564A; Sat, 15 Nov 2008 18:01:45 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from server.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id 572438FC19; Sat, 15 Nov 2008 18:01:45 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from localhost.corp.yahoo.com (john@localhost [IPv6:::1]) (authenticated bits=0) by server.baldwin.cx (8.14.3/8.14.3) with ESMTP id mAFI1NmT015493; Sat, 15 Nov 2008 13:01:38 -0500 (EST) (envelope-from jhb@freebsd.org) From: John Baldwin To: Won De Erick Date: Sat, 15 Nov 2008 11:25:44 -0500 User-Agent: KMail/1.9.7 References: <704830.24415.qm@web45815.mail.sp1.yahoo.com> <200811131438.25904.jhb@freebsd.org> <829178.39837.qm@web45811.mail.sp1.yahoo.com> In-Reply-To: <829178.39837.qm@web45811.mail.sp1.yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200811151125.44733.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (server.baldwin.cx [IPv6:::1]); Sat, 15 Nov 2008 13:01:39 -0500 (EST) X-Virus-Scanned: ClamAV 0.93.1/8636/Sat Nov 15 00:05:47 2008 on server.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.6 required=4.2 tests=AWL,BAYES_00,NO_RELAYS autolearn=ham version=3.1.3 X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on server.baldwin.cx Cc: Jeremy Chadwick , freebsd-hardware@freebsd.org Subject: Re: IRQ31 and IRQ32 on HPDL585 running FreeBSD 7.0 are consuming HIGH CPU usage X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Nov 2008 18:01:45 -0000 On Friday 14 November 2008 03:48:48 am Won De Erick wrote: > > > ----- Original Message ---- > > From: John Baldwin > > To: freebsd-hardware@freebsd.org > > Cc: Jeremy Chadwick ; Won De Erick > > Sent: Friday, November 14, 2008 3:38:25 AM > > Subject: Re: IRQ31 and IRQ32 on HPDL585 running FreeBSD 7.0 are consuming HIGH CPU usage > > > > On Thursday 13 November 2008 03:19:36 am Jeremy Chadwick wrote: > > > On Thu, Nov 13, 2008 at 12:07:37AM -0800, Won De Erick wrote: > > > > Noted on this, I will update you through this thread. > > > > > > > > However is there any possibility of the following: > > > > > > > > > I don't know if there's a way to split the interrupt request for each > bce's Rx and Tx, > > > > > which means a total of four IRQs, and eventually four cores (or 4 CPUs) > > > > > for the transactions. With this way, the IDLE processors would be > utilized. > > > > > > > > What I mean here is, for the two interfaces: > > > > > > > > one IRQ for bce0 Rx > > > > one IRQ for bce0 Tx > > > > one IRQ for bce1 Rx > > > > one IRQ for bce1 Tx > > > > > > I can't even begin to imagine how this would be possible on any NIC. > > > igb(4) does it. It is quite possible and one of the purposes of MSI. > > However, the current bce(4) hardware does not support this. It only allows > > for a single message and thus a single IRQ per-device. > > based from the man pages, igb driver supports Intel NICs w/ controllers starting from Intel NIC controller 82574. > > One Intel NIC (controller: 82576, see http://www.intel.com/Assets/PDF/prodbrief/320116.pdf) says it supports MSIX which minimizes the overhead of interrupts and allows load balancing of interrupt handling between multiple cores/CPUs. > > I should want a little more explanation how this feature being handled by MSIX. Thanks a lot. MSI/MSI-X is just an alternate facility PCI devices can use to assert interrupts rather than the legacy INTx lines. MSI/MSI-X support things like multiple messages per-device (rather than INTx where each function can only use a single intpin). As far as how a given PCI device makes use of MSI, that is up to that device's design and the driver. You would need to talk to the igb(4) maintainer to find out more details on how igb(4) uses MSI/MSI-X. -- John Baldwin From owner-freebsd-hardware@FreeBSD.ORG Sat Nov 15 20:48:49 2008 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CCDD9106568B for ; Sat, 15 Nov 2008 20:48:49 +0000 (UTC) (envelope-from sebosik@demax.sk) Received: from mail.demax.sk (mail.demax.sk [213.215.102.234]) by mx1.freebsd.org (Postfix) with ESMTP id 89DA78FC18 for ; Sat, 15 Nov 2008 20:48:49 +0000 (UTC) (envelope-from sebosik@demax.sk) Received: from localhost (localhost [127.0.0.1]) by mail.demax.sk (Postfix) with ESMTP id 97982873D9 for ; Sat, 15 Nov 2008 21:31:49 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at mail.demax.sk Received: from mail.demax.sk ([127.0.0.1]) by localhost (mail.demax.sk [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id x7u21UBGXzJh for ; Sat, 15 Nov 2008 21:31:49 +0100 (CET) Received: from [192.168.0.2] (2D204.demax.sk [195.62.17.204]) by mail.demax.sk (Postfix) with ESMTP id 71363873D8 for ; Sat, 15 Nov 2008 21:31:48 +0100 (CET) Message-ID: <491F31B5.90403@demax.sk> Date: Sat, 15 Nov 2008 21:31:49 +0100 From: Jan Sebosik User-Agent: Thunderbird 2.0.0.17 (Windows/20080914) MIME-Version: 1.0 To: freebsd-hardware@freebsd.org Content-Type: text/plain; charset=ISO-8859-2; format=flowed Content-Transfer-Encoding: 7bit Subject: Frebsd 7-STABLE, atapicd, atapicam and Intel errors X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Nov 2008 20:48:49 -0000 Hi all OS: Freebsd 7-STABLE from CVS of today Problematic HW: Intel DQ45CB (Q45 chipset, ICH10, SATA in native mode, but not AHCI), LG SATA DVD-RW GH20NS15 Problem: if I run freebsd without LG DVD connected to any SATA port onboard everything works allright. When I connect SATA DVD to board, then freebsd refuses to boot with messages similar to those: acd0: FAILURE-READ_BIG timed out unknown: FAILURE-READ_BIG timed out cddone: goit error 0x5 back Messages are repeating forever. Anybody knows where should be a bug? Temporary I`ve disconnected LG SATA burner from system board. Thanks for any idea. Best regards -- Jan Sebosik, Slovakia sebosik@demax.sk