From owner-freebsd-ipfw@FreeBSD.ORG Mon Oct 13 21:53:28 2014 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C2210720 for ; Mon, 13 Oct 2014 21:53:28 +0000 (UTC) Received: from m.nyi.net (m.nyi.net [66.111.12.250]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 97D0D65D for ; Mon, 13 Oct 2014 21:53:28 +0000 (UTC) Received: from m.nyi.net (localhost [127.0.0.1]) by m.nyi.net (Postfix) with ESMTP id B9936108997 for ; Mon, 13 Oct 2014 17:46:20 -0400 (EDT) Received: from m.nyi.net ([127.0.0.1]) by m.nyi.net (m.nyi.net [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Enq8rFI8SyzY for ; Mon, 13 Oct 2014 17:46:12 -0400 (EDT) Received: from [10.50.50.227] (urchin.nyi.net [64.147.100.2]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: jacob@nyi.net) by m.nyi.net (Postfix) with ESMTPSA id 06B70108994 for ; Mon, 13 Oct 2014 17:46:11 -0400 (EDT) Message-ID: <543C4825.6030901@nyi.net> Date: Mon, 13 Oct 2014 17:46:13 -0400 From: Jack Barber Organization: New York Internet Company User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.1.2 MIME-Version: 1.0 To: freebsd-ipfw@freebsd.org Subject: FreeBSD max pipe size? Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Oct 2014 21:53:28 -0000 I am trying to set up dummynet with FreeBSD 9.3 and a 10 GB Fibre over ethernet NIC (ix drivers). Dummynet appears to have a limit of 1.25 gigabits a second, and when I start setting extremely large pipe values I start running into: ipfw: bandwidth too large It doesn't like(in /etc/ipfw.rules) $IPFWcmd pipe 10 config bw 8000Mbits but for some reason it likes $IPFWcmd pipe 20 config bw 100000Mbits and $IPFWcmd pipe 10 config bw 10000Mbits even if there is no notcable performance diffrence betwen them $ iperf -c load-server2 ------------------------------------------------------------ Client connecting to load-server2, TCP port 5001 TCP window size: 434 KByte (default) ------------------------------------------------------------ [ 3] local 192.168.20.25 port 56122 connected with 192.168.20.20 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.0 sec 1.57 GBytes 1.35 Gbits/sec $ iperf -c load-server1 ------------------------------------------------------------ Client connecting to load-server1, TCP port 5001 TCP window size: 459 KByte (default) ------------------------------------------------------------ [ 3] local 192.168.20.20 port 49028 connected with 192.168.20.25 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.0 sec 1.36 GBytes 1.17 Gbits/sec testing is done on a private subnet which houses nothing more than two loading servers and a FreeBSD bridge in between. All machines have 10 GB/sec ix driver cards. Now, same set up, but commenting out all dummynet refrences in ipfw.rules: $ iperf -c load-server1 ------------------------------------------------------------ Client connecting to load-server1, TCP port 5001 TCP window size: 459 KByte (default) ------------------------------------------------------------ [ 3] local 192.168.20.20 port 49029 connected with 192.168.20.25 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.0 sec 8.44 GBytes 7.24 Gbits/sec $ iperf -c load-server2 ------------------------------------------------------------ Client connecting to load-server2, TCP port 5001 TCP window size: 434 KByte (default) ------------------------------------------------------------ [ 3] local 192.168.20.25 port 56123 connected with 192.168.20.20 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.0 sec 7.89 GBytes 6.78 Gbits/sec