Date: Wed, 21 Nov 2012 09:36:48 +0100 From: Andre Oppermann <andre@freebsd.org> To: Luigi Rizzo <rizzo@iet.unipi.it> Cc: Barney Cordoba <barney_cordoba@yahoo.com>, Adrian Chadd <adrian@freebsd.org>, khatfield@socllc.net, "freebsd-net@freebsd.org" <freebsd-net@freebsd.org>, Jim Thompson <jim@netgate.com>, Alfred Perlstein <bright@mu.org> Subject: Re: FreeBSD boxes as a 'router'... Message-ID: <50AC92A0.2000406@freebsd.org> In-Reply-To: <CA%2BhQ2%2BiBLi%2B3_GAVBfiTMmiZPnqGRomXYABawPct4n7-Nrq2Ew@mail.gmail.com> References: <1353448328.76219.YahooMailClassic@web121602.mail.ne1.yahoo.com> <E1F4816E-676C-4630-9FA1-817F737D007D@netgate.com> <50AC08EC.8070107@mu.org> <832757660.33924.1353460119408@238ae4dab3b4454b88aea4d9f7c372c1.nuevasync.com> <CAJ-Vmok8Ybdi%2BY8ZguMTKC7%2BF5=OxVDog27i4UgY-s3MCZkGcQ@mail.gmail.com> <250266404.35502.1353464214924@238ae4dab3b4454b88aea4d9f7c372c1.nuevasync.com> <50AC8393.3060001@freebsd.org> <CA%2BhQ2%2BiBLi%2B3_GAVBfiTMmiZPnqGRomXYABawPct4n7-Nrq2Ew@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 21.11.2012 09:04, Luigi Rizzo wrote: > On Tue, Nov 20, 2012 at 11:32 PM, Andre Oppermann <andre@freebsd.org> wrote: > >> On 21.11.2012 03:16, khatfield@socllc.net wrote: >> >>> I may be misstating. >>> >>> Specifically under high burst floods either routed or being dropped by pf >>> we would see the system >>> go unresponsive to user-level applications / SSH for example. >>> >>> The system would still function but it was inaccessible. To clarify as >>> well this was any number >>> of floods or attacks to any ports, the behavior remained. (These were not >>> SSH ports being hit) >>> >> >> I'm working on a hybrid interrupt/polling with live-lock prevention >> scheme in my svn branch. It works with a combination of disabling >> interrupts in interrupt context and then having an ithread loop over >> the RX DMA queue until it reaches the hardware and is done. Only >> then interrupts are re-enabled again. On a busy system it may never >> go back to interrupt. To prevent live-lock the ithread gives up the >> CPU after a normal quantum to let other threads/processes run as well. >> After that it gets immediately re-scheduled again with a sufficient >> high priority not get starved out by userspace. >> >> > very cool. this seems similar to NAPI. I've heard about NAPI but haven't looked at it. So I'm not sure how it works internally. In this case no special logic or kernel API support is required. Every driver can be converted and immediately use it. > The only adjustment i'd suggest to your scheme, if possible, is to add > some control (as it existed in the old polling architecture) to make sure > that userspace is not starved by the ithreads and other kernel tasks > (otherwise you can have livelock, as it happens with NAPI). > I am afraid that simple priorities do not work, you either need some > kind of fair scheduler, or put some hard limit on the cpu fraction used > by the kernel tasks when there are userspace processes around. That's the quantum stuff I talked about. When the ithread has used up its CPU quantum it gives up the CPU but schedules itself again at the same time. Also contrary to its name the ithread does not run in interrupt context but as a normal kernel thread with elevated priority. I have to thank Attilio though for a detailed explanation of what is going on behind the scenes in the interrupt and scheduler code to get a grip at the optimal approach for network drivers. The bus_setup_intr(9) and ithread documentation is very poor and misleading at the moment. I intend to fix that soon. -- Andre
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?50AC92A0.2000406>