Date: Tue, 28 Oct 2008 16:13:13 +0000 From: Bartosz Giza <gizmen@blurp.pl> To: freebsd-net@freebsd.org Cc: Alexander Motin <mav@freebsd.org> Subject: Re: two NIC on 2 core system (scheduling problem) Message-ID: <200810281613.13719.gizmen@blurp.pl> In-Reply-To: <49072BC9.4010103@FreeBSD.org> References: <1225203780.00029971.1225190402@10.7.7.3> <49072BC9.4010103@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tuesday 28 of October 2008 15:12:09 Alexander Motin wrote: > Bartosz Giza wrote: > > On other router based on the same hardware and software i have something > > like that: > > > > 10 root 1 171 ki31 0K 8K RUN 1 235.4H 78.66% idle: > > cpu1 11 root 1 171 ki31 0K 8K RUN 0 185.2H 72.12% idle: > > cpu0 20 root 1 -68 - 0K 8K - 0 48.7H 23.00% em0 > > taskq 23 root 1 -68 - 0K 8K WAIT 0 19.2H 9.67% > > irq16: fxp1 21 root 1 -68 - 0K 8K WAIT 1 28.2H 8.01% > > irq17: bge0 > > > > I don't know why on this router system balance over two cores. One > > difference is that on this router i have another fxp card (3 total) > > In verbose boot messages system shows that different IRQs assigned to > different APICs in round-robin fashion. So I may assume that this > IRQ->CPU mapping is static. em0's taskqueue at the same time able to > migrate CPUs as any usual process. I am not sure it is that true. When i watch top i see that sometimes irq: bge0 process switch to core 0 but most of the time is on core 1. So scheduler sometimes put this process on different core. > > Another question is why em0 taskq is eating so much cpu ? BGE interface > > is actually one that pushes 2 times more packets than em0 and it uses > > about half cpu comparing to em0. Is that not strange ? > > Could someone tell my why is this happening ? BGE is faster ? or maybe i > > can tune some > > The CPU time you see there includes much more then just a card handling > itself. It also includes CPU time of the most parts of network stack > used to process received packet. So if you have NAT, big firewall, > netgraph or any other CPU-hungry actions done with packets incoming via > em0 you will see such results. > Even more interesting is that if bge0 or fxp0 cards will require much > CPU time to send packet, this time will also be accounted to em0 process. > :) WOW that is weird. Yes i have quite big ipfw firewall and also couple of rules from pf. So you are saying that whole overhead from firewall is counted to this em taskq process. This is really strange for somebody who don't know about this. So what in case if i would have two em nic's. How would then overhead from firewalls be counted ? Splited to two taskq processes ? And another really weird thing is that if i have other card their processing are counted to tasq process of different NIC. Why is this done in such a way ?
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200810281613.13719.gizmen>