Date: Mon, 13 Dec 2004 22:34:18 +0100 From: Andre Oppermann <andre@freebsd.org> To: Andrew Gallatin <gallatin@cs.duke.edu> Cc: freebsd-net@freebsd.org Subject: Re: Rewritten TCP reassembly Message-ID: <41BE0ADA.9A1EE748@freebsd.org> References: <41BA0088.9000107@freebsd.org> <16830.1004.797220.47672@grasshopper.cs.duke.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
Andrew Gallatin wrote: > > Andre Oppermann writes: > > > > I've got some excellent review feedback from Mike Spengler and he found > > a off-by-one queue limit tracking error. > > > > http://www.nrg4u.com/freebsd/tcp_reass-20041213.patch > > > > Here are the same tests running your new patch in comparison to a > stock 6.x kernel from Friday. It looks like everything is faster. Excellent! > I've never seen 3.7Gb/sec on FreeBSD before. Nice work! Thanks! > The "before" and "tcp_reass-20041213" kernels differ only in the > contents of the above patch. I ran netperf with 20 times for each of > 4 socket buffer sizes (64KB, 128KB, 256KB, and 1MB) before and after > patching. All tests were run with net.isr.enable=1, and > machdep.cpu_idle_hlt=0. CPU was pretty much maxed thoughout. (SMP > kernel, 1 HTT p4). All the numbers in Mb/sec. The sender was running > linux-2.6.6 (also SMP on an identical HTT P4). > > The out-of-order accounting bug seems to be gone. Perfect. I want to get this patch into the tree next. I have already the next round in the works which is optimized even more by merging consecutive mbuf chains together (at the moment I have packet segment chains which have a direct pointer to the mbuf at the end of the chain) and which get passed in one go to soappend_stream. This removes the "present" loop and simplifies the general code a bit more again. With this and two other optimizations I have in mind you should be able to get very close to the theoretical maximum bandwidth of your current 4Gig Myrinet cards. There are a couple of other TCP tweaks that would help your special case some more now though. -- Andre
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?41BE0ADA.9A1EE748>