Date: Fri, 11 Apr 2014 19:05:55 +0300 From: Sami Halabi <sodynet1@gmail.com> To: Dennis Yusupoff <dyr@smartspb.net> Cc: "freebsd-net@freebsd.org" <freebsd-net@freebsd.org> Subject: Re: dummynet/ipfw high load? Message-ID: <CAEW%2BogaMpYTTLd8k-LfQFaiPerBWvvbH13=41AQRNw6Cgz8ohw@mail.gmail.com> In-Reply-To: <5347AEAA.9090801@smartspb.net> References: <5347AEAA.9090801@smartspb.net>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi, I had similar problem on the past and it turned to be the ammount of rules in ipfe. Using reduced subset with tables actually reduced the load. Sami =E2=80=8F=D7=91=D7=AA=D7=90=D7=A8=D7=99=D7=9A =D7=99=D7=95=D7=9D =D7=A9=D7= =99=D7=A9=D7=99, 11 =D7=91=D7=90=D7=A4=D7=A8=D7=99=D7=9C 2014, Dennis Yusup= off <dyr@smartspb.net> =D7=9B=D7=AA=D7=91: > Good day, gurus! > > We have a servers on the FreeBSD. They do NAT, shaping and traffic > accounting for our home (mainly) customers. > NAT realized with pf nat, shaping with ipfw dummynet and traffic > accounting with ng_netflow via ipfw ng_tee. > The problem is performance on (relatively) high traffic. > On Xeon E3-1270, whereas use Intel 10Gbit/sec 82599-based NIC(ix) or > Intel I350 (82579) in lagg transit traffic in 800 Mbit/sec and 100 kpps > [to customers] cause CPU load almost at 100% by interrupts from NIC or, > in case of net.isr.dispatch=3Ddeferred and net.inet.ip.fastforwarding=3D0= . > Deleting ipfw pipe decrease load at ~30% per cpu. > Deleting ipfw ng_tee (to ng_netflow) decrease load at 15% per cpu. > Turning off ipfw (sysctl net.inet.ip.fw.enable=3D0) decrease load more, s= o > what server can pass (nat'ed!) traffic on 1600 Mbit/sec and 200 kpps > with only load ~40% per cpu. > > So my questions are: > 1. Are there any way to decrease system load caused by dummynet/ipfw? > 2. Why dummynet/ipfw increase *interrupts* load, not kernel or > something like that? > 3. Are there any way to profiling that kind of load? Existing DTrace > and pmcstat examples almost useless or I just doesn't know how to do it > properly. > > Huge size of debugging info (including dtrace and pmcstat samples), > sysctl settings and so on, I opened appropriate topic at russian network > operator's forum: http://forum.nag.ru/forum/index.php?showtopic=3D93674 > In english it's available via google translate: > > http://translate.google.com/translate?hl=3Den&sl=3Dauto&tl=3Den&u=3Dhttp%= 3A%2F%2Fforum.nag.ru%2Fforum%2Findex.php%3Fshowtopic%3D93674 > > Feel free to ask me any question and do actions on the server! > > I would be VERY appreciate for any help and can take any measuring and > debugging on the one server. Moreover, I'm ready to give root access to > any of the appropriate person (as I already did it to Gleb Smirnoff when > we were investigate pf state problem). > > > -- > Best regards, > Dennis Yusupoff, > network engineer of > Smart-Telecom ISP > Russia, Saint-Petersburg > > _______________________________________________ > freebsd-net@freebsd.org <javascript:;> mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org<jav= ascript:;> > " > --=20 Sami Halabi Information Systems Engineer NMS Projects Expert FreeBSD SysAdmin Expert
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAEW%2BogaMpYTTLd8k-LfQFaiPerBWvvbH13=41AQRNw6Cgz8ohw>