Date: Thu, 9 May 2013 09:25:20 -0700 (PDT) From: Barney Cordoba <barney_cordoba@yahoo.com> To: =?iso-8859-1?Q?Cl=E9ment_Hermann_=28nodens=29?= <nodens2099@gmail.com>, Eugene Grosbein <egrosbein@rdtc.ru> Cc: freebsd-net@freebsd.org Subject: Re: High CPU interrupt load on intel I350T4 with igb on 8.3 Message-ID: <1368116720.51479.YahooMailClassic@web121606.mail.ne1.yahoo.com> In-Reply-To: <518BB8EE.2080705@rdtc.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
=0A=0A--- On Thu, 5/9/13, Eugene Grosbein <egrosbein@rdtc.ru> wrote:=0A=0A>= From: Eugene Grosbein <egrosbein@rdtc.ru>=0A> Subject: Re: High CPU interr= upt load on intel I350T4 with igb on 8.3=0A> To: ""Cl=E9ment Hermann (noden= s)"" <nodens2099@gmail.com>=0A> Cc: freebsd-net@freebsd.org=0A> Date: Thurs= day, May 9, 2013, 10:55 AM=0A> On 26.04.2013 18:31, "Cl=E9ment=0A> Hermann = (nodens)" wrote:=0A> > Hi list,=0A> > =0A> > We use pf+ALTQ for trafic shap= ing on some routers.=0A> > =0A> > We are switching to new servers : Dell Po= werEdge R620=0A> with 2 8-cores =0A> > Intel Processor (E5-2650L), 8GB RAM = and Intel I350T4=0A> (quad port) using =0A> > igb driver. The old hardware = is using em driver, the=0A> CPU load is high =0A> > but mostly due to kerne= l and a large pf ruleset.=0A> > =0A> > On the new hardware, we see high CPU= Interrupt load (up=0A> to 95%), even =0A> > though there is not much trafi= c currently (peaks about=0A> 150Mbps and =0A> > 40Kpps). All queues are use= d and binded to a cpu=0A> according to top, but a =0A> > lot of CPU time is= spent on igb queues (interrupt or=0A> wait). The load is =0A> > fine when = we stay below 20Kpps.=0A> > =0A> > We see no mbuf shortage, no dropped pack= et, but there=0A> is little margin =0A> > left on CPU time (about 25% idle = at best, most of CPU=0A> time is spent on =0A> > interrupts), which is dist= urbing.=0A> =0A> It seems you suffer from pf lock contention. You should st= op=0A> using pf=0A> with multi-core systems with 8.3. Move to ipfw+dummynet= or=0A> ng_car for 8.3=0A> or move to 10.0-CURRENT having new, rewritten pf= that does=0A> not have this problem.=0A> =0A> Network device driver is not= guilty here, that's just pf's=0A> contention=0A> running in igb's context.= =0A> =0A> Eugene Grosbein=0A=0AThey're both at play. Single threadedness ag= gravates subsystems that =0Ahave too many lock points.=0A=0AIt can also be = "solved" with using 1 queue, because then you don't=0Ahave 4 queues going i= nto a single thread.=0A=0ABC
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1368116720.51479.YahooMailClassic>