Date: Mon, 13 Jul 2009 02:20:32 -0700 (PDT) From: Barney Cordoba <barney_cordoba@yahoo.com> To: freebsd-current@freebsd.org, Chris Buechler <cmb@pfsense.org> Subject: Re: Flowtables -- any tuning hints? Message-ID: <837660.83798.qm@web63903.mail.re1.yahoo.com>
next in thread | raw e-mail | index | archive | help
=0A=0A--- On Sun, 7/12/09, Chris Buechler <cmb@pfsense.org> wrote:=0A=0A> F= rom: Chris Buechler <cmb@pfsense.org>=0A> Subject: Re: Flowtables -- any tu= ning hints?=0A> To: freebsd-current@freebsd.org=0A> Date: Sunday, July 12, = 2009, 6:41 PM=0A> Kip Macy wrote:=0A> > On Sat, Jul 11, 2009 at 10:24 AM, S= cott Ullrich<sullrich@gmail.com>=0A> wrote:=0A> >=A0=A0=A0=0A> >> Hello Fre= ebsd-current@ folks,=0A> >> =0A> >> I see with the commit "svn commit: r191= 259 -=0A> head/sys/netinet"=0A> >> flowtables have been added..=A0 Cool!=0A= > >> =0A> >> Does anyone have any tuning hints for this=0A> addition -- spe= cifically=0A> >> how much memory does the hash table=0A> consume?=A0=A0=A0O= r better yet does any=0A> >> documentation exist for this newly added featu= re?=0A> >> =0A> >> Looking for an easy way to calculate max flows for=0A> t= he amount of=0A> >> memory installed in a FreeBSD machine.=0A> >>=A0 =A0=A0= =A0=0A> > =0A> > =0A> > You want to avoid hash collisions. So, generally=0A= > speaking you want the=0A> > hash table to be sized 2x larger than the num= ber of=0A> unique connection=0A> > destinations.=A0 You want the maximum nu= mber of=0A> flows to be as large as=0A> > the maximum number of unique dest= inations x number of=0A> cores. When you=0A> > get to the case of hundreds = of thousands of unique=0A> destinations as in=0A> > the case of a small ISP= doing IP forwarding, you're=0A> probably better=0A> > off disabling the fl= owtable. For most other workloads=0A> its likely to be=0A> > a clear win. R= unning a process on an 8-core system=0A> with 8 threads each=0A> > calling = sendto(...) with 10 bytes I can push 3.5 -=0A> 4Mpps (with cxgb -=0A> > you= won't get this with most cards) with the flowtable=0A> enabled. With=0A> >= the flowtable disabled lock contention causes=0A> performance to degrade= =0A> > to 330kpps with the aforementioned workload.=0A> >=A0=A0=A0=0A> This= is interesting functionality, but I think we need to=0A> look at it a bit = closer for our use case. Is there any=0A> benefit in running this in a fire= wall scenario? That's=0A> primarily what Scott and I (pfsense) are interest= ed in. In=0A> our world, if you're pushing 50Kpps+, you're almost=0A> certa= inly falling into the "small ISP doing IP forwarding"=0A> scenario with hun= dreds of thousands of unique destinations.=0A> Where we usually see these k= inds of loads are small ISPs,=0A> web hosting companies, or universities (w= hich are=0A> functionally not much diff from a small ISP), all of which=0A>= I'm familiar with falling into the "better off disabling"=0A> category. I = also suspect pf's locking negates some or all of=0A> the benefits here.=0A>= =0A> I suspect it's not applicable to the specific workload our=0A> users = normally have, where you're almost entirely doing IP=0A> forwarding, and in= itiating very little if any traffic. bz@=0A> said it's not something you wa= nt on a router. Is that a fair=0A> assessment? =0A> best,=0A> Chris=0A> =0A= =0AI don't see the need for flowtables or any sort of hashing with=0Amulti = queue boards in a pass-through environment (such as a firewall). =0AThe flo= ws are already hashed by the hardware so doing all of the =0Agobbledygook i= n flowtables seems like a waste of cycles. Just matching=0Ayour receive and= transmit queues in a properly designed driver is all=0Athats necessary for= lock avoidance.=0A=0ABarney=0A=0A=0A
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?837660.83798.qm>