From owner-freebsd-ipfw@FreeBSD.ORG Fri Jan 16 01:51:13 2004 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F0C8316A4CE for ; Fri, 16 Jan 2004 01:51:12 -0800 (PST) Received: from holodoc.ip.se (ua-213-115-163-137.cust.bredbandsbolaget.se [213.115.163.137]) by mx1.FreeBSD.org (Postfix) with ESMTP id A514043D69 for ; Fri, 16 Jan 2004 01:51:04 -0800 (PST) (envelope-from rd@tilde.se) Received: by holodoc.ip.se (Postfix, from userid 103) id 37235128454; Fri, 16 Jan 2004 10:48:41 +0100 (CET) Received: from nyalaptopen (c-f79572d5.02-85-73746f13.cust.bredbandsbolaget.se [213.114.149.247]) by holodoc.ip.se (Postfix) with ESMTP id 20F6812844E for ; Fri, 16 Jan 2004 10:48:36 +0100 (CET) Message-ID: <009f01c3dc16$3b9270f0$7901010a@nyalaptopen> From: "Rickard Dahlstrand" To: References: <001b01c3daea$49b70740$7901010a@nyalaptopen> Date: Fri, 16 Jan 2004 10:50:52 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1158 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 X-Sanitizer: This message has been sanitized! Subject: Re: Dummynet performance X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jan 2004 09:51:13 -0000 Hi, Never mind this. After recompiling the kernel and adding a bunch of new parameters I got less than or 1 ms delay and up to 65mbit/s TCP bandwidth. (Up to 93mbit/s with a larger window-size!!) For anybody trying the same, try doing a new kernel with this configuration: options BRIDGE options IPFIREWALL options IPDIVERT options IPFIREWALL_VERBOSE options IPFW2 options HZ=2000 options DUMMYNET options DEVICE_POLLING and add this to /boot/loader.conf kern.ipc.nmbclusters="32768" kern.ipc.nsfbufs="32768" and to /etc/sysctl.conf net.link.ether.bridge_cfg=fxp1:0,fxp2:0 net.link.ether.bridge_ipfw=1 net.link.ether.bridge=1 net.inet.tcp.delayed_ack=0 By the way, this is a 1Ghz Via machine and I get approx 20% cpu load getting thru 93% TCP traffic. Memory is contant at 91M free of 128M and no swap. FreeBSD rocks!! Rickard. ----- Original Message ----- From: "Rickard Dahlstrand" To: Sent: Wednesday, January 14, 2004 11:03 PM Subject: Dummynet performance Hi, I have a fresh FreeBSD 4.9 installation and have activated the ipfw and dummynet kernel modules and configured two ports as a bridge. Everything works fine except one thing. When I add a pipe that does any kind of bw limiting, I can't get thru more than 5 mbit/s TCP on my bridged interface. UDP and packets with larger tcp windows size work better. One issue I found was the RTT. It's <1ms in all cases except when I add bw limiting, then it's up to 15-20ms. Does the bw limiting add this much RTT? Here is the config and iperf results. ipfw -q flush ipfw -q add allow ip from any to any via fxp0 ipfw -q add pipe 1 all from any to any via fxp1 in ipfw -q add pipe 2 all from any to any via fxp2 in ipfw -q pipe 1 config bw 20Mbit/s delay 0ms ipfw -q pipe 2 config bw 20Mbit/s delay 0ms >iperf -c 10.1.1.100 ------------------------------------------------------------ Client connecting to 10.1.1.100, TCP port 5001 TCP window size: 8.00 KByte (default) ------------------------------------------------------------ [1952] local 10.1.1.121 port 3540 connected with 10.1.1.100 port 5001 [ ID] Interval Transfer Bandwidth [1952] 0.0-10.0 sec 6.70 MBytes 5.60 Mbits/sec >iperf -c 10.1.1.100 -w 100k ------------------------------------------------------------ Client connecting to 10.1.1.100, TCP port 5001 TCP window size: 100 KByte ------------------------------------------------------------ [1952] local 10.1.1.121 port 3518 connected with 10.1.1.100 port 5001 [ ID] Interval Transfer Bandwidth [1952] 0.0-10.0 sec 17.8 MBytes 14.9 Mbits/sec If I do this: ipfw -q flush ipfw -q add allow ip from any to any via fxp0 ipfw -q add pipe 1 all from any to any via fxp1 in ipfw -q add pipe 2 all from any to any via fxp2 in ipfw -q pipe 1 config ipfw -q pipe 2 config I get this: >iperf -c 10.1.1.100 ------------------------------------------------------------ Client connecting to 10.1.1.100, TCP port 5001 TCP window size: 8.00 KByte (default) ------------------------------------------------------------ [1952] local 10.1.1.121 port 3541 connected with 10.1.1.100 port 5001 [ ID] Interval Transfer Bandwidth [1952] 0.0-10.0 sec 108 MBytes 90.8 Mbits/sec Another issue I have is the fact that the delay-function doesn't appear to work. (Or am I doing something wrong) ipfw -q flush ipfw -q add allow ip from any to any via fxp0 ipfw -q add pipe 1 all from any to any via fxp1 in ipfw -q add pipe 2 all from any to any via fxp2 in ipfw -q pipe 1 config delay 10ms ipfw -q pipe 2 config delay 10ms Reply from 10.1.1.100: bytes=32 time=11ms TTL=128 Reply from 10.1.1.100: bytes=32 time=15ms TTL=128 Reply from 10.1.1.100: bytes=32 time=13ms TTL=128 Reply from 10.1.1.100: bytes=32 time=12ms TTL=128 Ping statistics for 10.1.1.100: Packets: Sent = 4, Received = 4, Lost = 0 (0% loss), Approximate round trip times in milli-seconds: Minimum = 11ms, Maximum = 15ms, Average = 12ms If I normally have <1ms to this host and according to the config I should have 20ms. I would really like some help on these issues. Regards, Rickard.