From owner-freebsd-performance@FreeBSD.ORG  Mon Feb  4 18:42:55 2008
Return-Path: <owner-freebsd-performance@FreeBSD.ORG>
Delivered-To: freebsd-performance@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 4158616A41A
	for <freebsd-performance@freebsd.org>;
	Mon,  4 Feb 2008 18:42:55 +0000 (UTC)
	(envelope-from thompsa@FreeBSD.org)
Received: from heff.fud.org.nz (203-109-251-39.static.bliink.ihug.co.nz
	[203.109.251.39])
	by mx1.freebsd.org (Postfix) with ESMTP id DFD1713C442
	for <freebsd-performance@freebsd.org>;
	Mon,  4 Feb 2008 18:42:54 +0000 (UTC)
	(envelope-from thompsa@FreeBSD.org)
Received: by heff.fud.org.nz (Postfix, from userid 1001)
	id 8A6227776; Tue,  5 Feb 2008 07:29:45 +1300 (NZDT)
Date: Tue, 5 Feb 2008 07:29:45 +1300
From: Andrew Thompson <thompsa@FreeBSD.org>
To: Stefan Lambrev <stefan.lambrev@moneybookers.com>
Message-ID: <20080204182945.GA49276@heff.fud.org.nz>
References: <4794E6CC.1050107@moneybookers.com>
	<47A0B023.5020401@moneybookers.com>
	<m21w7x5ilg.wl%gnn@neville-neil.com>
	<47A3074A.3040409@moneybookers.com>
	<47A72EAB.6070602@moneybookers.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <47A72EAB.6070602@moneybookers.com>
User-Agent: Mutt/1.5.16 (2007-06-09)
X-Mailman-Approved-At: Mon, 04 Feb 2008 19:09:43 +0000
Cc: freebsd-performance@freebsd.org
Subject: Re: network performance
X-BeenThere: freebsd-performance@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Performance/tuning <freebsd-performance.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-performance>,
	<mailto:freebsd-performance-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-performance>
List-Post: <mailto:freebsd-performance@freebsd.org>
List-Help: <mailto:freebsd-performance-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-performance>,
	<mailto:freebsd-performance-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 04 Feb 2008 18:42:55 -0000

On Mon, Feb 04, 2008 at 05:26:35PM +0200, Stefan Lambrev wrote:
> Greetings,
>
> In my desire to increase network throughput, and to be able to handle more 
> then ~250-270kpps
> I started experimenting with lagg and link aggregation control protocol 
> (lacp).
> To my surprise this doesn't increase the amount of packets my server can 
> handle
>
> Using lagg doesn't improve situation at all, and also errors are not 
> reported.
> Also using lagg increased content switches:
>
> Top showed for CPU states +55%   system, which is quite high?
>
> I'll use hwpmc and lock_profiling to see where the kernel spends it's time.

Thanks for investigating this. One thing to note is that ip flows from
the same connection always go down the same interface, this is because
Ethernet is not allowed to reorder frames. The hash uses
src-mac, dst-mac, src-ip and dst-ip (see lagg_hashmbuf), make sure when
performance testing that your traffic varies in these values. Adding
tcp/udp ports to the hashing may help.

I look forward to your profiling results.


cheers,
Andrew