From owner-freebsd-net@FreeBSD.ORG  Fri Apr 25 14:02:29 2014
Return-Path: <owner-freebsd-net@FreeBSD.ORG>
Delivered-To: freebsd-net@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115])
 (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTPS id ADE6316C
 for <freebsd-net@freebsd.org>; Fri, 25 Apr 2014 14:02:29 +0000 (UTC)
Received: from smtp1.multiplay.co.uk (smtp1.multiplay.co.uk [85.236.96.35])
 by mx1.freebsd.org (Postfix) with ESMTP id 6DF1217E2
 for <freebsd-net@freebsd.org>; Fri, 25 Apr 2014 14:02:29 +0000 (UTC)
Received: by smtp1.multiplay.co.uk (Postfix, from userid 65534)
 id A937120E7088B; Fri, 25 Apr 2014 14:02:27 +0000 (UTC)
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on
 smtp1.multiplay.co.uk
X-Spam-Level: **
X-Spam-Status: No, score=2.2 required=8.0 tests=AWL,BAYES_00,DOS_OE_TO_MX,
 FSL_HELO_NON_FQDN_1,HELO_NO_DOMAIN,RDNS_DYNAMIC,STOX_REPLY_TYPE autolearn=no
 version=3.3.1
Received: from r2d2 (82-69-141-170.dsl.in-addr.zen.co.uk [82.69.141.170])
 by smtp1.multiplay.co.uk (Postfix) with ESMTPS id 0371820E70886;
 Fri, 25 Apr 2014 14:02:23 +0000 (UTC)
Message-ID: <4E7A280CACC84910A3DE068085976E30@multiplay.co.uk>
From: "Steven Hartland" <killing@multiplay.co.uk>
To: "Marek Salwerowicz" <marek_sal@wp.pl>,
 =?iso-8859-1?Q?Gerrit_K=FChn?= <gerrit.kuehn@aei.mpg.de>
References: <535A1354.2040309@wp.pl>
 <20140425113711.e7c7d1c2.gerrit.kuehn@aei.mpg.de> <535A482E.1030106@wp.pl>
 <20140425140123.a76c18f9.gerrit.kuehn@aei.mpg.de> <535A5268.100@wp.pl>
 <8247FE6336414E1F97ADA561D0680097@multiplay.co.uk> <535A5DD9.9060206@wp.pl>
 <AC1233D368D94519BB59B484D6F385C0@multiplay.co.uk> <535A69CE.9010800@wp.pl>
Subject: Re: NFS over LAGG / lacp poor performance
Date: Fri, 25 Apr 2014 15:02:15 +0100
MIME-Version: 1.0
Content-Type: text/plain; format=flowed; charset="iso-8859-1";
 reply-type=original
Content-Transfer-Encoding: 8bit
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 6.00.2900.5931
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157
Cc: freebsd-net@freebsd.org
X-BeenThere: freebsd-net@freebsd.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net/>
List-Post: <mailto:freebsd-net@freebsd.org>
List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 25 Apr 2014 14:02:29 -0000


----- Original Message ----- 
From: "Marek Salwerowicz" <marek_sal@wp.pl>
To: "Steven Hartland" <killing@multiplay.co.uk>; "Gerrit Kühn" <gerrit.kuehn@aei.mpg.de>
Cc: <freebsd-net@freebsd.org>
Sent: Friday, April 25, 2014 2:57 PM
Subject: Re: NFS over LAGG / lacp poor performance


>W dniu 2014-04-25 15:27, Steven Hartland pisze:
>> In that case I believe you've hard coded the number of queues, check
>> /boot/loader.conf
>> for references to this.
>
> Yes, that's true:
>
> % cat /boot/loader.conf
> debug.acpi.max_tasks="128"
> if_lagg_load="YES"
> kern.ipc.nmbclusters="131072"
> hw.igb.num_queues="1"
>
>
> I am wondering  why I have turned this on..
> My box has raidz2 ZFS box with SSD mirrored log..

We find that large numbers of queues causes high interrupt issues
however at a guess you did this to enable the machine to boot with
all nics due to lack of auto mbuf tuning in 9.x.

I'd go with ~2 queues per nic.

    Regards
    Steve