Date: Wed, 29 Oct 2008 09:53:28 +0100 From: Bartosz Giza <gizmen@blurp.pl> To: freebsd-net@freebsd.org Cc: Alexander Motin <mav@freebsd.org> Subject: Re: two NIC on 2 core system (scheduling problem) Message-ID: <200810290953.28237.gizmen@blurp.pl> In-Reply-To: <490755A3.4050903@FreeBSD.org> References: <1225203780.00029971.1225190402@10.7.7.3> <200810281613.13719.gizmen@blurp.pl> <490755A3.4050903@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Tuesday 28 of October 2008 19:10:43 Alexander Motin napisa=C5=82(a): > Bartosz Giza wrote: > >> The CPU time you see there includes much more then just a card > >> handling itself. It also includes CPU time of the most parts of > >> network stack used to process received packet. So if you have NAT, big > >> firewall, netgraph or any other CPU-hungry actions done with packets > >> incoming via em0 you will see such results. > >> Even more interesting is that if bge0 or fxp0 cards will require much > >> CPU time to send packet, this time will also be accounted to em0 > >> process. I have checked this and you are right. When i turned off ipfw; taskq proces= s=20 started to use less cpu. But still what is strange why processing from=20 other cards are counted in em0 taskq ? This is quite strange and in that=20 way em0 taskq process is using more cpu on one of the cores. So what i=20 think the best would be to have only em NICs because processing of the=20 packets would be splitted to those taskq processes is that right ? > > WOW that is weird. Yes i have quite big ipfw firewall and also couple > > of rules from pf. So you are saying that whole overhead from firewall > > is counted to this em taskq process. This is really strange for > > somebody who don't know about this. > > So what in case if i would have two em nic's. How would then overhead > > from firewalls be counted ? Splited to two taskq processes ? > > And another really weird thing is that if i have other card their > > processing are counted to tasq process of different NIC. Why is this > > done in such a way ? > > There is no dedicated processes in system to handle routing, firewall, > netgraph, etc. Many processes would lead to multiple context switches, > bigger latencies and reduced performance. Instead most parts of network > stack implemented in direct call fashion. So NIC receive interrupt > initiates packet handling by doing stack function call, that function > calls another and so on until packet will be completely processed and > transmitted or queued. There is not so many exception from this rule, > for example, dummynet which schedules part of activity to the timer > events. So sometimes you still can see some activity from the swi:net, > dummynet or some other threads. It also tells not so much about who > really consumed the CPU, but more about who initiated that CPU > consumption. Ok, good to know. But how is counted firewall overhead when i would have=20 only bge cards. They don't use taskq so i assume i would see this as system= =20 usage correct ? > In case of two em NICs em0 thread will mostly show load produced by > em0->em1 traffic processing and em1 - load produced by em1->em0 traffic.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200810290953.28237.gizmen>