From owner-freebsd-performance@FreeBSD.ORG Tue May 11 13:51:10 2010 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id F4220106564A; Tue, 11 May 2010 13:51:09 +0000 (UTC) (envelope-from gallatin@cs.duke.edu) Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by mx1.freebsd.org (Postfix) with ESMTP id B66AA8FC19; Tue, 11 May 2010 13:51:09 +0000 (UTC) Received: from grapeape2.cs.duke.edu (grapeape2.cs.duke.edu [152.3.140.76]) by duke.cs.duke.edu (8.14.2/8.14.2) with ESMTP id o4BDp5kb008584 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 11 May 2010 09:51:05 -0400 (EDT) X-DKIM: Sendmail DKIM Filter v2.8.3 duke.cs.duke.edu o4BDp5kb008584 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=cs.duke.edu; s=mail; t=1273585865; bh=sJ4HXaoJGcjlwosClQLRaemaLSaYjhPgnfgE2LGqOrw=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:In-Reply-To; b=dtnsbOENfuHYNaPeHg1mDSYHIrdCEn4Ttv+4rx8aqMWrH+JJLnfY2S/8QKJ9onsap GjqGz/cHztHdxjBgeMBahxB365e75b22fh4HH2B6q8oyjW4SOejRsFaOpu12exM+O9 wso4eQP0nRwhWIBjvVtFQGi2QVjLwaARmNi2CgnY= Received: (from gallatin@localhost) by grapeape2.cs.duke.edu (8.12.10/8.12.10/Submit) id o4BDp3nH029426; Tue, 11 May 2010 09:51:03 -0400 (EDT) Date: Tue, 11 May 2010 09:51:03 -0400 From: Andrew Gallatin To: Murat Balaban Message-ID: <20100511135103.GA29403@grapeape2.cs.duke.edu> References: <4BE52856.3000601@unsane.co.uk> <1273323582.3304.31.camel@efe> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1273323582.3304.31.camel@efe> X-Operating-System: SunOS 5.10 on an sun4u User-Agent: Mutt/1.5.13 (2006-08-11) Cc: freebsd-net@freebsd.org, freebsd-performance@freebsd.org Subject: Re: Intel 10Gb X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 May 2010 13:51:10 -0000 Murat Balaban [murat@enderunix.org] wrote: > > Much of the FreeBSD networking stack has been made parallel in order to > cope with high packet rates at 10 Gig/sec operation. > > I've seen good numbers (near 10 Gig) in my tests involving TCP/UDP > send/receive. (latest Intel driver). > > As far as BPF is concerned, above statement does not hold true, > since there is some work that needs to be done here in terms > of BPF locking and parallelism. My tests show that there > is a high lock contention around "bpf interface lock", resulting > in input errors at high packet rates and with many bpf devices. If you're interested in 10GbE packet sniffing at line rate on the cheap, have a look at the Myri10GE "sniffer" interface. This is a special software package that takes a normal mxge(4) NIC, and replaces the driver/firmware with a "myri_snf" driver/firmware which is optimized for packet sniffing. Using this driver/firmware combo, we can receive minimal packets at line rate (14.8Mpps) to userspace. You can even access this using a libpcap interface. The trick is that the fast paths are OS-bypass, and don't suffer from OS overheads, like lock contention. See http://www.myri.com/scs/SNF/doc/index.html for details. Best Regards, Drew