Date: Fri, 4 Jul 2008 16:28:47 +1000 (EST) From: Bruce Evans <brde@optusnet.com.au> To: Paul <paul@gtcomm.net> Cc: FreeBSD Net <freebsd-net@freebsd.org>, Ingo Flaschberger <if@xip.at> Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] Message-ID: <20080704160950.Y9864@delplex.bde.org> In-Reply-To: <486DAD0D.8090604@gtcomm.net> References: <4867420D.7090406@gtcomm.net> <alpine.LFD.1.10.0807010257570.19444@filebunker.xip.at> <20080701010716.GF3898@stlux503.dsto.defence.gov.au> <alpine.LFD.1.10.0807010308320.19444@filebunker.xip.at> <486986D9.3000607@monkeybrains.net> <48699960.9070100@gtcomm.net> <ea7b9c170806302005n2a66f592h2127f87a0ba2c6d2@mail.gmail.com> <20080701033117.GH83626@cdnetworks.co.kr> <ea7b9c170806302050p2a3a5480t29923a4ac2d7c852@mail.gmail.com> <4869ACFC.5020205@gtcomm.net> <4869B025.9080006@gtcomm.net> <486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net> <486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net> <alpine.LFD.1.10.0807021052041.557@filebunker.xip.at> <486B4F11.6040906@gtcomm.net> <alpine.LFD.1.10.0807021155280.557@filebunker.xip.at> <486BC7F5.5070604@gtcomm.net> <20080703160540.W6369@delplex.bde.org> <486C7F93.7010308@gtcomm.net> <20080703195521.O6973@delplex.bde. org> <486D35A0.4000302@gtcomm.net> <486DAD0D.8090604@gtcomm.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, 4 Jul 2008, Paul wrote: > Numbers are maximum with near 100% cpu usage and some errors occuring, just > for testing. > FreeBSD 7.0-STABLE FreeBSD 7.0-STABLE #6: Thu Jul 3 19:32:38 CDT 2008 > root@foo:/usr/obj/usr/src/sys/ROUTER amd64 > CPU: Dual-Core AMD Opteron(tm) Processor 2222 (3015.47-MHz K8-class CPU) > NON-SMP KERNEL em driver, intel 82571EB NICs > fastforwarding on, isr.direct on, ULE, Preemption (NOTE: Interesting thing, > without preemption gets errors similar to polling) PREEMPTION is certainly needed with UP. Without it, interrupts don't actually work (to work, they need to preempt the running thread, but they often (usually?) don't do that). Then with UP, there is a good chance that the interrupt thread doesn't get scheduled to run for a long time, but with SMP (especially with lots of CPUs) there is a good chance that another CPU gets scheduled to run the interrupt thread. em (unless misconfigured) doesn't have an interrupt thread; it uses a taskq which might take even longer to be scheduled than an interrupt thread. I use PREEMPTION with UP and !PREEMPTION with SMP. With polling, missed polls cause the same packet loss as not preempting. > I tried polling, and I tried the polling patch that was posted to the list > and both work but generate too many errors (missed packets). > Without polling the packet errors ONLY occur when the cpu is near 100% usage Polling should also only cause packet loss when the CPU is near 100% usage, but now transients of near 100% usually cause packet loss, while with interrupts it takes a transient of > 100% on the competing interrupt- driven resources to cause packet loss. Pleas trim quotes. Bruce
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20080704160950.Y9864>