Date: Wed, 15 Mar 2006 23:48:07 +1300 From: Andrew Thompson <thompsa@freebsd.org> To: Chih-Chang Hsieh <cch@kmu.edu.tw> Cc: freebsd-net@freebsd.org Subject: Re: if_bridge + polling get lower througphts Message-ID: <20060315104807.GA20361@heff.fud.org.nz> In-Reply-To: <4417E97D.7050202@kmu.edu.tw> References: <4417E97D.7050202@kmu.edu.tw>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, Mar 15, 2006 at 06:16:29PM +0800, Chih-Chang Hsieh wrote: > We have a FreeBSD 6.1-PRELEASE box. > > It runs with 2 em NICs and uses if_bridge + IPFW + ipfilter + pf. > > This box usually gets a very high interrupt rate >90%. > > By using netstat -I em0 1, we see: > > input (em0) output > packets errs bytes packets errs bytes colls > 15164 0 9549410 16842 0 12132262 0 > 15331 0 9220723 17466 0 12354486 0 > 16486 0 10089165 17926 0 13256014 0 > 14936 10 8363433 17090 0 13367271 0 > 14712 0 8449705 16701 0 13443079 0 > > > When we enable the polling on em0 and em1 by ifconfig, netstat -I em0 shows: > > input (em0) output > packets errs bytes packets errs bytes colls > 10910 0 6629176 11467 0 7681346 0 > 10951 0 6916042 11397 0 7954205 0 > 11281 0 7437813 10804 0 7390386 0 > 10711 0 6308033 11605 0 8400405 0 > 10807 0 6270716 10973 0 7700748 0 > > the interrupt rate decreases (<30%) but we get lower throuphts. > > net.link.bridge.ipfw: 1 > net.link.bridge.pfil_member: 1 > net.link.bridge.pfil_bridge: 1 > net.link.bridge.pfil_onlyip: 0 > > How could we tune the box to reach higher throughputs In the above setup you are running four packet filters, ipfilter, pf and IPFW twice (ipfw=1 is an extra layer2 run). More specifically the packet filters are actually invoked _10_ times per packet, in_if -> bridge_if -> out_if per pfil_hook, plus the extra ipfw layer2. This explains your high cpu usage, you may want to reorganise the filtering to optimise it. Andrew
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20060315104807.GA20361>