From owner-freebsd-stable@FreeBSD.ORG Thu Jan 27 17:41:52 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 111FD10656A4 for ; Thu, 27 Jan 2011 17:41:52 +0000 (UTC) (envelope-from jack.vogel@intel.com) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by mx1.freebsd.org (Postfix) with ESMTP id DEABB8FC25 for ; Thu, 27 Jan 2011 17:41:51 +0000 (UTC) Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga102.fm.intel.com with ESMTP; 27 Jan 2011 09:41:51 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="4.60,386,1291622400"; d="scan'208";a="881583920" Received: from orsmsx603.amr.corp.intel.com ([10.22.226.49]) by fmsmga001.fm.intel.com with ESMTP; 27 Jan 2011 09:41:49 -0800 Received: from orsmsx601.amr.corp.intel.com (10.22.226.213) by orsmsx603.amr.corp.intel.com (10.22.226.49) with Microsoft SMTP Server (TLS) id 8.2.254.0; Thu, 27 Jan 2011 09:41:49 -0800 Received: from orsmsx508.amr.corp.intel.com ([10.22.226.46]) by orsmsx601.amr.corp.intel.com ([10.22.226.213]) with mapi; Thu, 27 Jan 2011 09:41:49 -0800 From: "Vogel, Jack" To: Damien Fleuriot , Jeremy Chadwick Date: Thu, 27 Jan 2011 09:41:48 -0800 Thread-Topic: High interrupt rate on a PF box + performance Thread-Index: Acu+SBzCXp13vjqUT1K7RqlhPscCTAAAJ4FA Message-ID: <1DB50624F8348F48840F2E2CF6040A9D014BEB8833@orsmsx508.amr.corp.intel.com> References: <4D41417A.20904@my.gd> <20110127172724.GA36587@icarus.home.lan> <4D41ABF1.1010405@my.gd> In-Reply-To: <4D41ABF1.1010405@my.gd> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "freebsd-stable@freebsd.org" , "freebsd-pf@freebsd.org" Subject: RE: High interrupt rate on a PF box + performance X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jan 2011 17:41:52 -0000 Jeremy is right, if you have a problem the first step is to try the latest = code. However, when I look at the interrupts below I don't see what the problem i= s? The Broadcom seems to have about the same rate, it just doesn't have MSIX (= multiple vectors). Jack -----Original Message----- From: Damien Fleuriot [mailto:ml@my.gd]=20 Sent: Thursday, January 27, 2011 9:31 AM To: Jeremy Chadwick Cc: freebsd-stable@freebsd.org; freebsd-pf@freebsd.org; Vogel, Jack Subject: Re: High interrupt rate on a PF box + performance On 1/27/11 6:27 PM, Jeremy Chadwick wrote: > On Thu, Jan 27, 2011 at 10:57:14AM +0100, Damien Fleuriot wrote: >> Hello list, >> >> I have a problem with interrupts, network cards, and PF performance. >> >> We have 2 firewalls running FreeBSD 8.0 for the current master and >> FreeBSD 8.1 for the backup host, which I upgraded just yesterday. >> >> [...] >> >> vmstat -i >> --- >> interrupt total rate >> irq14: ata0 36 0 >> irq16: mfi0 353244 1 >> irq21: uhci0 uhci+ 461504 1 >> cpu0: timer 615183815 1996 >> irq256: bce0 1015412475 3295 >> irq257: igb0 1067318584 3464 >> irq258: igb0 695648752 2258 >> irq259: igb0 2 0 >> irq260: igb1 11503857 37 >> irq261: igb1 506598 1 >> irq262: igb1 69 0 >> irq269: bce1 790820 2 >> cpu1: timer 615183757 1996 >> cpu2: timer 615197165 1996 >> cpu3: timer 615197165 1996 >> Total 5252757843 17050 >=20 > There are changes to the igb(4) driver which are in RELENG_8 (8-STABLE), > and some which will be in the upcoming 8.2-RELEASE, which may address > this. Jack Vogel of Intel would be able to confirm for sure; CC'ing him > here. >=20 > Could you please provide output from the following commands? >=20 > * pciconf -lvcb (only include igbX entries, thanks) > * sysctl -a | grep msi >=20 > Thanks. >=20 > I can't help with the CARP-related issues or other stuff you're > experiencing. These issues may all be separate problems, hard to say. >=20 igb0@pci0:14:0:0: class=3D0x020000 card=3D0x145a8086 chip=3D0x10d68086 rev=3D0x02 hdr=3D0x00 igb1@pci0:14:0:1: class=3D0x020000 card=3D0x145a8086 chip=3D0x10d68086 rev=3D0x02 hdr=3D0x00 igb2@pci0:15:0:0: class=3D0x020000 card=3D0x145a8086 chip=3D0x10d68086 rev=3D0x02 hdr=3D0x00 igb3@pci0:15:0:1: class=3D0x020000 card=3D0x145a8086 chip=3D0x10d68086 rev=3D0x02 hdr=3D0x00 hw.bce.msi_enable: 1 hw.pci.honor_msi_blacklist: 1 hw.pci.enable_msix: 1 hw.pci.enable_msi: 1 igb0: flags=3D8943 metric 0 mtu 1500 options=3D13b ether 00:1b:21:12:ec:38 inet [snip] netmask 0xffffffc0 broadcast [snip] media: Ethernet autoselect (1000baseT ) status: active igb1: flags=3D8843 metric 0 mtu 150= 0 options=3D13b ether 00:1b:21:12:ec:39 inet 10.0.0.252 netmask 0xffffff00 broadcast 10.0.0.255 media: Ethernet autoselect (1000baseT ) status: active Here you go :) Note that the igb2 and 3 interfaces are unused, unplugged, unconfigured