From owner-freebsd-stable@FreeBSD.ORG Thu Jan 27 20:58:47 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6D04A1065672 for ; Thu, 27 Jan 2011 20:58:47 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta09.emeryville.ca.mail.comcast.net (qmta09.emeryville.ca.mail.comcast.net [76.96.30.96]) by mx1.freebsd.org (Postfix) with ESMTP id 4B5E78FC0A for ; Thu, 27 Jan 2011 20:58:46 +0000 (UTC) Received: from omta10.emeryville.ca.mail.comcast.net ([76.96.30.28]) by qmta09.emeryville.ca.mail.comcast.net with comcast id 0kut1g0040cQ2SLA9kymcP; Thu, 27 Jan 2011 20:58:46 +0000 Received: from koitsu.dyndns.org ([98.248.34.134]) by omta10.emeryville.ca.mail.comcast.net with comcast id 0kyl1g00d2tehsa8WkylyL; Thu, 27 Jan 2011 20:58:46 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 44AB09B422; Thu, 27 Jan 2011 12:58:45 -0800 (PST) Date: Thu, 27 Jan 2011 12:58:45 -0800 From: Jeremy Chadwick To: Damien Fleuriot Message-ID: <20110127205845.GA41537@icarus.home.lan> References: <4D41417A.20904@my.gd> <1DB50624F8348F48840F2E2CF6040A9D014BEB8833@orsmsx508.amr.corp.intel.com> <4D41B197.6070308@my.gd> <201101280146.57028.wmn@siberianet.ru> <4D41C9FC.10503@my.gd> <20110127195741.GA40449@icarus.home.lan> <4D41D7BE.3030208@my.gd> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4D41D7BE.3030208@my.gd> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Sergey Lobanov , "freebsd-stable@freebsd.org" , "freebsd-pf@freebsd.org" Subject: Re: High interrupt rate on a PF box + performance X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jan 2011 20:58:47 -0000 On Thu, Jan 27, 2011 at 09:38:22PM +0100, Damien Fleuriot wrote: > On 1/27/11 8:57 PM, Jeremy Chadwick wrote: > <...snipping out stuff...> > We're also considering moving to faster machines but I don't think that > will help much with our problem. > > I suppose additional CPU cores will be of no help at all, considering > the kernel is single threaded and runs on cpu0 only ? Kernel folks should be able to talk about this in detail, but my understanding is that the kernel itself supports multiple threads, but the question is whether or not the drivers or relevant "pieces" (e.g. igb(4) driver, pf, TCP stack, etc.) support SMP (multi-core/threading) or not. I think this is referred to as something being "MPSAFE" or not. The things you see during boot -- [ITHREAD], [FILTER], and [GIANT-LOCKED] play a role as well, but I think those indicate what style of locking is used (since some drivers/features might not work properly in a multiprocessor environment). I'm trying to avoid correlating "multiprocessor safe" with "makes use of multiple processors". I'm an old 65xxx CPU guy, this SMP stuff is still "new technology" to me when it comes to actual operations/mechanics. Regarding TCP and SMP, this is regularly touched on in the FreeBSD Status Reports that go out (always worth reading). See "TCP SMP scalability project": http://www.freebsd.org/news/status/report-2010-10-2010-12.html I know all this information is technical of course and doesn't answer your question directly. I wish there was something more authoritative when it came to this question. > Actually, I assume it might even be detrimental to us to add more cores, > since they'll spend more time interrupting each other ? > > Thanks for sharing your thoughts :) -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP 4BD6C0CB |