From owner-freebsd-net@FreeBSD.ORG Fri Apr 25 13:06:37 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B56F0853 for ; Fri, 25 Apr 2014 13:06:37 +0000 (UTC) Received: from mx4.wp.pl (mx4.wp.pl [212.77.101.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.wp.pl", Issuer "Thawte SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 371131189 for ; Fri, 25 Apr 2014 13:06:36 +0000 (UTC) Received: (wp-smtpd smtp.wp.pl 19382 invoked from network); 25 Apr 2014 15:06:32 +0200 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wp.pl; s=1024a; t=1398431192; bh=FuslYgNyY9CygrIReoNvM79EN5PWotnIhWtPeWl+aCQ=; h=From:To:CC:Subject; b=AVXG/xK6H5opF3tbnF5NnRxq8eXsnRgPGujkKsCjkzAVTRBQoaQAOWXm4s97Xl7tc 7dpFWmpfA/VPhU+MVrJ+vS2ISzH+cNF6usSQEcVcZmtdRcmO0YZNhc72lM2Zf3Kevt a4VDZzkJtB+6P4a/p0e53VqBMjWe+gHeOVlVnud0= Received: from pb-d-128-141-237-169.cern.ch (HELO [128.141.237.169]) (marek_sal@[128.141.237.169]) (envelope-sender ) by smtp.wp.pl (WP-SMTPD) with AES128-SHA encrypted SMTP for ; 25 Apr 2014 15:06:32 +0200 Message-ID: <535A5DD9.9060206@wp.pl> Date: Fri, 25 Apr 2014 15:06:33 +0200 From: Marek Salwerowicz User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Steven Hartland , =?ISO-8859-1?Q?Gerrit_K=FC?= =?ISO-8859-1?Q?hn?= Subject: Re: NFS over LAGG / lacp poor performance References: <535A1354.2040309@wp.pl> <20140425113711.e7c7d1c2.gerrit.kuehn@aei.mpg.de> <535A482E.1030106@wp.pl> <20140425140123.a76c18f9.gerrit.kuehn@aei.mpg.de> <535A5268.100@wp.pl> <8247FE6336414E1F97ADA561D0680097@multiplay.co.uk> In-Reply-To: <8247FE6336414E1F97ADA561D0680097@multiplay.co.uk> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-WP-AV: skaner antywirusowy poczty Wirtualnej Polski S. A. X-WP-SPAM: NO 0000000 [YUMk] Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 13:06:37 -0000 W dniu 2014-04-25 14:55, Steven Hartland pisze: > ----- Original Message ----- From: "Marek Salwerowicz" > > >> W dniu 2014-04-25 14:01, Gerrit Kühn pisze: >>> Thanks for your input. As far as I understood so far, there should >>> be one >>> igb queue created per cpu core in the system by default (and this is >>> what >>> I see on my system). But my irq rate looks quite high to me (and it is >>> only on one of these queues). >> >> >> My CPU has 8 cores: >> >> http://ark.intel.com/products/75267/Intel-Xeon-Processor-E5-2640-v2-20M-Cache-2_00-GHz >> >> >> So why do I have only 1 queue ? > > What does "sysctl hw.igb.num_queues" report? storage1% sysctl hw.igb.num_queues hw.igb.num_queues: 1 > > num_queues does default to 1 for Legacy or MSI so you might be hitting > that. > > Do you see "Using MSIX interrupts with" in your dmesg? storage% dmesg | grep MSIX igb0: Using MSIX interrupts with 2 vectors igb1: Using MSIX interrupts with 2 vectors igb2: Using MSIX interrupts with 2 vectors igb3: Using MSIX interrupts with 2 vectors igb0: Using MSIX interrupts with 2 vectors igb1: Using MSIX interrupts with 2 vectors igb2: Using MSIX interrupts with 2 vectors igb3: Using MSIX interrupts with 2 vectors -- Marek