From owner-freebsd-net@FreeBSD.ORG Fri Apr 25 11:40:56 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4BC3BCD0 for ; Fri, 25 Apr 2014 11:40:56 +0000 (UTC) Received: from mx4.wp.pl (mx4.wp.pl [212.77.101.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.wp.pl", Issuer "Thawte SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C3D2816C3 for ; Fri, 25 Apr 2014 11:40:55 +0000 (UTC) Received: (wp-smtpd smtp.wp.pl 26385 invoked from network); 25 Apr 2014 13:34:06 +0200 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wp.pl; s=1024a; t=1398425646; bh=3LO5PEXlZ5gP08lhQ9m9Lqz2kg0mA3TC9/2GgghCOhM=; h=From:To:CC:Subject; b=md3f79iurp6JYHMxpq11QaFLp/aRd3FWEFkBs15PG4O6ZgvDD9OKxwg4SoFd7K8BD VwvXdYPtga/qUo09D3+7PaIhsvpSBXKRGQFsnjg5TQD+++dunEjLQt61C/uGdkGSQZ Mc6SJCblBz4RBtxKzDu+TNGgQ8hbs7dB8JfZIpro= Received: from pb-d-128-141-237-169.cern.ch (HELO [128.141.237.169]) (marek_sal@[128.141.237.169]) (envelope-sender ) by smtp.wp.pl (WP-SMTPD) with AES128-SHA encrypted SMTP for ; 25 Apr 2014 13:34:06 +0200 Message-ID: <535A482E.1030106@wp.pl> Date: Fri, 25 Apr 2014 13:34:06 +0200 From: Marek Salwerowicz User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: =?ISO-8859-1?Q?Gerrit_K=FChn?= Subject: Re: NFS over LAGG / lacp poor performance References: <535A1354.2040309@wp.pl> <20140425113711.e7c7d1c2.gerrit.kuehn@aei.mpg.de> In-Reply-To: <20140425113711.e7c7d1c2.gerrit.kuehn@aei.mpg.de> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-WP-AV: skaner antywirusowy poczty Wirtualnej Polski S. A. X-WP-SPAM: NO 0000000 [cUOE] Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 11:40:56 -0000 Hello Gerrit, W dniu 2014-04-25 11:37, Gerrit Kühn pisze: > On Fri, 25 Apr 2014 09:48:36 +0200 Marek Salwerowicz > wrote about NFS over LAGG / lacp poor performance: > > > > Could you have a look on your system and let me know how your interrupts > are spread? For me on storage1 (9.1-RELEASE) it looks like: > storage1% vmstat > -i > > interrupt total rate > irq1: atkbd0 77 0 > irq16: ehci0 11355459 1 > irq23: ehci1 26468060 3 > cpu0:timer 565127221 79 > irq264: mps0 1905530055 267 > irq265: igb0:que 0 2307223482 323 > irq266: igb0:link 4 0 > irq267: igb1:que 0 271641638 38 > irq268: igb1:link 6 0 > irq269: igb2:que 0 91665104 12 > irq270: igb2:link 6 0 > irq271: igb3:que 0 628139928 88 > irq272: igb3:link 5 0 > irq273: ahci0 1579878 0 > cpu1:timer 283555294 39 > cpu4:timer 285185117 40 > cpu10:timer 305482789 42 > cpu3:timer 239294364 33 > cpu7:timer 406658435 57 > cpu2:timer 353421522 49 > cpu8:timer 437284694 61 > cpu11:timer 261929365 36 > cpu5:timer 282975629 39 > cpu6:timer 248355888 34 > cpu9:timer 256679915 36 > Total 9169553935 1287 > marek@storage1:/home/marek% But in my case all igb links are aggregated using LACP (lagg0), then there are 2 vlans over lagg0 (vlan14 and vlan900) and the vlan900 is one dedicated for NFS And I don't have more than one queue per interface Cheers, Marek -- Marek Salwerowicz