Date: Mon, 5 Mar 2007 13:52:28 -0500 From: John Baldwin <jhb@freebsd.org> To: Robert Watson <rwatson@freebsd.org> Cc: freebsd-net@freebsd.org, freebsd-current@freebsd.org, Andre Oppermann <andre@freebsd.org>, Andrew Gallatin <gallatin@cs.duke.edu>, kmacy@freebsd.org Subject: Re: New optimized soreceive_stream() for TCP sockets, proof of concept Message-ID: <200703051352.29939.jhb@freebsd.org> In-Reply-To: <20070305182755.S31701@fledge.watson.org> References: <45E8276D.60105@freebsd.org> <17900.24574.751134.397740@grasshopper.cs.duke.edu> <20070305182755.S31701@fledge.watson.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Monday 05 March 2007 13:29, Robert Watson wrote: > On Mon, 5 Mar 2007, Andrew Gallatin wrote: > > > With the patch, we finally seem to be performance competative on the receive > > side with Linux x86_64 and Solaris/amd64 on this same hardware. Both of > > those OSes do much better (saturate the link with jumbos) when CPU affinity > > is used to bind the interrupt handler and netserver process to different > > cores on the same socket. I imagine FreeBSD may be able to do even better > > if it ever grows CPU affinity support for both interrupt handlers and > > processes. With the patch, it performs at least as well, if not better > > than, Solaris and Linux do without CPU affinity. > > I don't have numbers in front of me, and am currently packing for a trip to > Tokyo so won't find them before traveling, but my experience has been that > binding the ithread to a specific CPU is very helpful in improving receive > performance. You can slap a sched_bind(0) into the interrupt handler the > first time it runs and it should stick appropriately, and add a sysctl to > sched_bind() for a user process as a hack to test it out. > > John has a patch that pins interrupt threads, etc, not sure what the status of > that is. CC'd. Tested and around for over a year. Sent to people several times but no benchmarking has resulted. It lives in p4 in //depot/user/jhb/intr/... I've just regenerated the patch at http://www.FreeBSD.org/~jhb/patches/intr_bind.patch (same URL as the last N times I've posted it). I do now have access again to the sources for a simple intr_bind userland tool that lets you move assign the CPU for a given interrupt. There is currently no way to see which CPU an interrupt is assigned to outside of 'show intr' in ddb or scraping a verbose dmesg, but that can always be added later. What is there now is probably sufficient for testing. -- John Baldwin
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200703051352.29939.jhb>