From owner-freebsd-net@FreeBSD.ORG Mon Dec 13 21:04:51 2004 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 43FBB16A4CE; Mon, 13 Dec 2004 21:04:51 +0000 (GMT) Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9A44043D31; Mon, 13 Dec 2004 21:04:50 +0000 (GMT) (envelope-from gallatin@cs.duke.edu) Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu [152.3.145.30]) by duke.cs.duke.edu (8.13.1/8.13.1) with ESMTP id iBDL4nIn021743 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 13 Dec 2004 16:04:49 -0500 (EST) Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.12.9p2/8.12.9/Submit) id iBDL4i4V054808; Mon, 13 Dec 2004 16:04:44 -0500 (EST) (envelope-from gallatin) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <16830.1004.797220.47672@grasshopper.cs.duke.edu> Date: Mon, 13 Dec 2004 16:04:44 -0500 (EST) To: Andre Oppermann In-Reply-To: <41BDD1C7.7060105@freebsd.org> References: <41BA0088.9000107@freebsd.org> <41BDD1C7.7060105@freebsd.org> X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid cc: freebsd-net@freebsd.org Subject: Re: Rewritten TCP reassembly X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Dec 2004 21:04:51 -0000 Andre Oppermann writes: > > I've got some excellent review feedback from Mike Spengler and he found > a off-by-one queue limit tracking error. > > http://www.nrg4u.com/freebsd/tcp_reass-20041213.patch > Here are the same tests running your new patch in comparison to a stock 6.x kernel from Friday. It looks like everything is faster. I've never seen 3.7Gb/sec on FreeBSD before. Nice work! The "before" and "tcp_reass-20041213" kernels differ only in the contents of the above patch. I ran netperf with 20 times for each of 4 socket buffer sizes (64KB, 128KB, 256KB, and 1MB) before and after patching. All tests were run with net.isr.enable=1, and machdep.cpu_idle_hlt=0. CPU was pretty much maxed thoughout. (SMP kernel, 1 HTT p4). All the numbers in Mb/sec. The sender was running linux-2.6.6 (also SMP on an identical HTT P4). The out-of-order accounting bug seems to be gone. Drew x before.65536 + tcp_reass-20041213.65536 +--------------------------------------------------------------------------+ | + | | + | | xx + + + + +*x + ++ ++ + ++ + +++| ||__________________A________|______M__|_____MA________________| | +--------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 4 3121.03 3135.88 3135.27 3128.3325 8.3667691 + 20 3126.75 3151.97 3139.41 3139.608 7.5192088 Difference at 95.0% confidence 11.2755 +/- 8.67923 0.360432% +/- 0.277439% (Student's t, pooled s = 7.64032) x before.131072 + tcp_reass-20041213.131072 +--------------------------------------------------------------------------+ | x x ++ xx + + + | |x x xxx xx*x x x+* + xxx++ + + + ++ + + + +| | |_________AM_______||_____________A_M_____________| | +--------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 20 3336.63 3516.95 3449.87 3447.838 52.708815 + 20 3443.67 3717.15 3579.92 3571.8355 80.445555 Difference at 95.0% confidence 123.997 +/- 43.5271 3.59638% +/- 1.26245% (Student's t, pooled s = 68.0063) x before.262144 + tcp_reass-20041213.262144 +--------------------------------------------------------------------------+ | x + | | x x ++ x + ++ | |+ x x x x xx+ *x x * ++ +xx xx+ x++ + * + +| | |_______|____A_________A__|M__________| | +--------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 20 3197.77 3421.67 3316.64 3314.8555 59.787371 + 20 3135.57 3479.29 3380.31 3361.5785 71.303501 Difference at 95.0% confidence 46.723 +/- 42.1136 1.4095% +/- 1.27045% (Student's t, pooled s = 65.7979) x before.1048576 + tcp_reass-20041213.1048576 +--------------------------------------------------------------------------+ | + + | | x + +x x+ x+ | |xx x x *x * x x * +x +* x+x+x++ x+ ++ +| | |_______________A|__M________AM|__________| | +--------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 20 2658.05 2732.45 2706.91 2700.1955 23.318833 + 20 2686.35 2765.88 2722.09 2720.7805 18.752084 Difference at 95.0% confidence 20.585 +/- 13.5427 0.762352% +/- 0.501546% (Student's t, pooled s = 21.159)