From owner-freebsd-ipfw@FreeBSD.ORG Mon Dec 18 01:47:41 2006 Return-Path: X-Original-To: freebsd-ipfw@freebsd.org Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D008716A403 for ; Mon, 18 Dec 2006 01:47:41 +0000 (UTC) (envelope-from bounces@nabble.com) Received: from talk.nabble.com (www.nabble.com [72.21.53.35]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9B13E43C9F for ; Mon, 18 Dec 2006 01:47:41 +0000 (GMT) (envelope-from bounces@nabble.com) Received: from [72.21.53.38] (helo=jubjub.nabble.com) by talk.nabble.com with esmtp (Exim 4.50) id 1Gw7M9-0002UA-BV for freebsd-ipfw@freebsd.org; Sun, 17 Dec 2006 17:31:53 -0800 Message-ID: <7922408.post@talk.nabble.com> Date: Sun, 17 Dec 2006 17:31:53 -0800 (PST) From: Looza To: freebsd-ipfw@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Nabble-From: elias_nogueira@hotmail.com Subject: problem simulating a LFN X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Dec 2006 01:47:41 -0000 Hi, I'm trying to simulate a LFN(Long Fat Network) to test various TCP versions. I am using iperf on the end PCs. I'm trying to use FreeBSD's ipfw to simulate a LFN. ex. throughput=1 Gbit/s and delay=100ms, trhoughput=100 Mbit/s and delay=1s This is how my hardware is connected. |Linux PC|--Gigabit--| FreeBSD box |--Gigabit--|Linux PC| With this configuration: >ipfw 500 add allow ip from any to any I get speeds of about 200 to 300 Mbit/s and the CPU usage goes up to 99%. I've googled this problem and it seems that some good quality NIC do most of the processing so that the CPU doesn't have to. Unfortunately I have bad quality NICs so the CPU does all the processing, stoping the throughput at these speeds. With this configuration: >ipfw 100 pipe 1 add ip from 192.168.0.1 to 192.168.1.1 >ipfw 500 add allow ip from any to any >ipfw pipe 1 config delay 100ms I get speed of just 10 Mbit/s. I've do some testing and increasing the delay will lower the troughput, decreasing th delay will boost the throughput. I've tested the queue and the bw option and it seems it makes no difference. PIng will show 200ms RTT(Round Trip Time) wich is what I'd expect except for the troughput. My first question is if anyone knows off some way to get the processing out of the CPU so that I can achieve gigabit speeds without ? My sencond question is if there is a way of setting the ipfw delay option so that it does not alter the troughput? PS - I hope I can be understood. -- View this message in context: http://www.nabble.com/problem-simulating-a-LFN-tf2837638.html#a7922408 Sent from the freebsd-ipfw mailing list archive at Nabble.com. From owner-freebsd-ipfw@FreeBSD.ORG Mon Dec 18 05:05:37 2006 Return-Path: X-Original-To: freebsd-ipfw@freebsd.org Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A687A16A407 for ; Mon, 18 Dec 2006 05:05:37 +0000 (UTC) (envelope-from infofarmer@gmail.com) Received: from nf-out-0910.google.com (nf-out-0910.google.com [64.233.182.189]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0D47F43C9F for ; Mon, 18 Dec 2006 05:04:20 +0000 (GMT) (envelope-from infofarmer@gmail.com) Received: by nf-out-0910.google.com with SMTP id x37so1737850nfc for ; Sun, 17 Dec 2006 21:04:12 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; b=n5vxD/FciCr3h2EwquwUBM0ppdBHOx23DsiIXASGcLcbpAgYdvWMYLd3ZyaUqw615EzwpPH44Mm+YwF0nOiw7xn2v+XZbM0Y4aHG5xMNZvbzVWnuk26S5bntKVVoLz25smoKJ+C+DvrUmaxuWa+R4m7QPiTkO/wK9wIXFKvVT1o= Received: by 10.78.166.7 with SMTP id o7mr2608373hue.1166416660318; Sun, 17 Dec 2006 20:37:40 -0800 (PST) Received: by 10.78.167.16 with HTTP; Sun, 17 Dec 2006 20:37:40 -0800 (PST) Message-ID: Date: Mon, 18 Dec 2006 07:37:40 +0300 From: "Andrew Pantyukhin" Sender: infofarmer@gmail.com To: Looza In-Reply-To: <7922408.post@talk.nabble.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <7922408.post@talk.nabble.com> X-Google-Sender-Auth: efe436986f2cd81e Cc: freebsd-ipfw@freebsd.org Subject: Re: problem simulating a LFN X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Dec 2006 05:05:37 -0000 On 12/18/06, Looza wrote: > My first question is if anyone knows off some way to > get the processing out of the CPU so that I can achieve > gigabit speeds without ? Like you already noticed, better NICs help. BTW, which ones do you have? See drivers' manpages for info about hardware offloading. Also, have a look at polling(4), it might (or might not) help. > My second question is if there is a way of setting the > ipfw delay option so that it does not alter the throughput? You'll have to read ipfw(8) for that. In short, you might have to increase the queue size so that it can hold 100ms worth of traffic. From owner-freebsd-ipfw@FreeBSD.ORG Mon Dec 18 06:59:59 2006 Return-Path: X-Original-To: freebsd-ipfw@freebsd.org Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 8454016A412 for ; Mon, 18 Dec 2006 06:59:59 +0000 (UTC) (envelope-from if@hetzner.co.za) Received: from mail1a.your-server.co.za (mail1a.your-server.co.za [196.7.18.227]) by mx1.FreeBSD.org (Postfix) with ESMTP id C85F243CB7 for ; Mon, 18 Dec 2006 06:59:23 +0000 (GMT) (envelope-from if@hetzner.co.za) Received: from [192.168.2.25] (helo=hetzner.co.za) by mail1a.your-server.co.za with esmtpa (Exim 4.63) (envelope-from ) id 1GwByj-0004D2-UL; Mon, 18 Dec 2006 08:28:01 +0200 Received: from localhost ([127.0.0.1]) by hetzner.co.za with esmtp (Exim 4.63 (FreeBSD)) (envelope-from ) id 1GwByh-0000mz-KP; Mon, 18 Dec 2006 08:27:59 +0200 To: Looza From: Ian FREISLICH In-Reply-To: Message from Looza of "Sun, 17 Dec 2006 17:31:53 PST." <7922408.post@talk.nabble.com> X-Attribution: BOFH Date: Mon, 18 Dec 2006 08:27:59 +0200 Message-Id: X-Authenticated-Sender: if@hetzner.co.za X-Virus-Scanned: Clear (ClamAV 0.88.4/2352/Sun Dec 17 22:22:59 2006) Cc: freebsd-ipfw@freebsd.org Subject: Re: problem simulating a LFN X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Dec 2006 06:59:59 -0000 Looza wrote: > > Hi, > > I'm trying to simulate a LFN(Long Fat Network) to test various TCP versions. > I am using iperf on the end PCs. I'm trying to use FreeBSD's ipfw to > simulate a LFN. > ex. throughput=1 Gbit/s and delay=100ms, trhoughput=100 Mbit/s and delay=1s > > This is how my hardware is connected. > > |Linux PC|--Gigabit--| FreeBSD box |--Gigabit--|Linux PC| > > > With this configuration: > > >ipfw 500 add allow ip from any to any > > I get speeds of about 200 to 300 Mbit/s and the CPU usage goes up > to 99%. I've googled this problem and it seems that some good > quality NIC do most of the processing so that the CPU doesn't have > to. Unfortunately I have bad quality NICs so the CPU does all the > processing, stoping the throughput at these speeds. Try sysctl net.inet.ip.fastforwarding=1 . With this set I can formard and filter in excess of 500kpps using less than 5% CPU on a Xeon. I highly recomend the Intel 82546GB chip based adaptors (em driver). Ian -- Ian Freislich From owner-freebsd-ipfw@FreeBSD.ORG Mon Dec 18 11:08:47 2006 Return-Path: X-Original-To: freebsd-ipfw@FreeBSD.org Delivered-To: freebsd-ipfw@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A2CFA16A40F for ; Mon, 18 Dec 2006 11:08:47 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [69.147.83.40]) by mx1.FreeBSD.org (Postfix) with ESMTP id 118D743CB5 for ; Mon, 18 Dec 2006 11:08:41 +0000 (GMT) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (linimon@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.4/8.13.4) with ESMTP id kBIB8T5I089933 for ; Mon, 18 Dec 2006 11:08:29 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.13.4/8.13.4/Submit) id kBIB8RnE089929 for freebsd-ipfw@FreeBSD.org; Mon, 18 Dec 2006 11:08:27 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 18 Dec 2006 11:08:27 GMT Message-Id: <200612181108.kBIB8RnE089929@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: linimon set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-ipfw@FreeBSD.org Cc: Subject: Current problem reports assigned to you X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Dec 2006 11:08:47 -0000 Current FreeBSD problem reports Critical problems Serious problems S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/51274 ipfw [ipfw] [patch] ipfw2 create dynamic rules with parent o kern/73910 ipfw [ipfw] serious bug on forwarding of packets after NAT o kern/74104 ipfw [ipfw] ipfw2/1 conflict not detected or reported, manp o conf/78762 ipfw [ipfw] [patch] /etc/rc.d/ipfw should excecute $firewal o bin/80913 ipfw [patch] /sbin/ipfw2 silently discards MAC addr arg wit o kern/88659 ipfw [modules] ipfw and ip6fw do not work properly as modul o kern/93300 ipfw ipfw pipe lost packets o kern/95084 ipfw [ipfw] [patch] IPFW2 ignores "recv/xmit/via any" (IPFW o kern/97504 ipfw [ipfw] IPFW Rules bug o kern/97951 ipfw [ipfw] [patch] ipfw does not tie interface details to o kern/98831 ipfw [ipfw] ipfw has UDP hickups o kern/102471 ipfw [ipfw] [patch] add tos and dscp support o kern/103454 ipfw [ipfw] [patch] add a facility to modify DF bit of the o kern/106534 ipfw [ipfw] [panic] ipfw + dummynet 14 problems total. Non-critical problems S Tracker Resp. Description -------------------------------------------------------------------------------- a kern/26534 ipfw [ipfw] Add an option to ipfw to log gid/uid of who cau o kern/46159 ipfw [ipfw] [patch] ipfw dynamic rules lifetime feature o kern/48172 ipfw [ipfw] [patch] ipfw does not log size and flags o bin/50749 ipfw [ipfw] [patch] ipfw2 incorrectly parses ports and port o kern/55984 ipfw [ipfw] [patch] time based firewalling support for ipfw o kern/60719 ipfw [ipfw] Headerless fragments generate cryptic error mes o kern/69963 ipfw [ipfw] install_state warning about already existing en o kern/71366 ipfw [ipfw] "ipfw fwd" sometimes rewrites destination mac a o kern/72987 ipfw [ipfw] ipfw/dummynet pipe/queue 'queue [BYTES]KBytes ( o kern/73276 ipfw [ipfw] [patch] ipfw2 vulnerability (parser error) o bin/78785 ipfw [ipfw] [patch] ipfw verbosity locks machine if /etc/rc o kern/80642 ipfw [ipfw] [patch] ipfw small patch - new RULE OPTION o kern/82724 ipfw [ipfw] [patch] Add setnexthop and defaultroute feature o kern/86957 ipfw [ipfw] [patch] ipfw mac logging o kern/87032 ipfw [ipfw] [patch] ipfw ioctl interface implementation o kern/91847 ipfw [ipfw] ipfw with vlanX as the device o kern/103328 ipfw sugestions about ipfw table o kern/104682 ipfw [ipfw] [patch] Some minor language consistency fixes a o bin/104921 ipfw [patch] ipfw(8) sometimes treats ipv6 input as ipv4 (a o kern/105330 ipfw [ipfw] [patch] ipfw (dummynet) does not allow to set q 20 problems total. From owner-freebsd-ipfw@FreeBSD.ORG Tue Dec 19 18:12:47 2006 Return-Path: X-Original-To: freebsd-ipfw@freebsd.org Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 313BF16A691 for ; Tue, 19 Dec 2006 18:12:47 +0000 (UTC) (envelope-from bounces@nabble.com) Received: from talk.nabble.com (www.nabble.com [72.21.53.35]) by mx1.FreeBSD.org (Postfix) with ESMTP id AE9C843CB0 for ; Tue, 19 Dec 2006 18:12:46 +0000 (GMT) (envelope-from bounces@nabble.com) Received: from [72.21.53.38] (helo=jubjub.nabble.com) by talk.nabble.com with esmtp (Exim 4.50) id 1GwjSH-0001sU-VR for freebsd-ipfw@freebsd.org; Tue, 19 Dec 2006 10:12:45 -0800 Message-ID: <7951880.post@talk.nabble.com> Date: Tue, 19 Dec 2006 10:12:45 -0800 (PST) From: Looza To: freebsd-ipfw@freebsd.org In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Nabble-From: elias_nogueira@hotmail.com References: <7922408.post@talk.nabble.com> Subject: Re: problem simulating a LFN X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Dec 2006 18:12:47 -0000 I used that option but it overrode the ipfw delay option. My second question was due to the TCP buffers, I had already configured them but the changes weren't permanent. Ian FREISLICH wrote: > > > Try sysctl net.inet.ip.fastforwarding=1 . With this set I can > formard and filter in excess of 500kpps using less than 5% CPU on > a Xeon. > > I highly recomend the Intel 82546GB chip based adaptors (em driver). > > Ian > > -- > Ian Freislich > _______________________________________________ > freebsd-ipfw@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-ipfw > To unsubscribe, send any mail to "freebsd-ipfw-unsubscribe@freebsd.org" > > -- View this message in context: http://www.nabble.com/problem-simulating-a-LFN-tf2837638.html#a7951880 Sent from the freebsd-ipfw mailing list archive at Nabble.com. From owner-freebsd-ipfw@FreeBSD.ORG Tue Dec 19 21:41:59 2006 Return-Path: X-Original-To: freebsd-ipfw@freebsd.org Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 151A216A403 for ; Tue, 19 Dec 2006 21:41:59 +0000 (UTC) (envelope-from bounces@nabble.com) Received: from talk.nabble.com (www.nabble.com [72.21.53.35]) by mx1.FreeBSD.org (Postfix) with ESMTP id CF4EF43C9F for ; Tue, 19 Dec 2006 21:41:58 +0000 (GMT) (envelope-from bounces@nabble.com) Received: from [72.21.53.38] (helo=jubjub.nabble.com) by talk.nabble.com with esmtp (Exim 4.50) id 1Gwmii-0007WD-3e for freebsd-ipfw@freebsd.org; Tue, 19 Dec 2006 13:41:56 -0800 Message-ID: <7955491.post@talk.nabble.com> Date: Tue, 19 Dec 2006 13:41:56 -0800 (PST) From: Looza To: freebsd-ipfw@freebsd.org In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Nabble-From: elias_nogueira@hotmail.com References: <7922408.post@talk.nabble.com> Subject: Re: problem simulating a LFN X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Dec 2006 21:41:59 -0000 About the offloading, it might be just that but I not sure and I don't have time to explore this situation. I'm using 2 SMC9452TX and 1 SMC9452TX-1. The delay issue was because of the TCP buffers, I had configured them but the changes weren't permanent. infofarmer wrote: > > On 12/18/06, Looza wrote: >> My first question is if anyone knows off some way to >> get the processing out of the CPU so that I can achieve >> gigabit speeds without ? > > Like you already noticed, better NICs help. BTW, which > ones do you have? See drivers' manpages for info about > hardware offloading. Also, have a look at polling(4), it > might (or might not) help. > >> My second question is if there is a way of setting the >> ipfw delay option so that it does not alter the throughput? > > You'll have to read ipfw(8) for that. In short, you might > have to increase the queue size so that it can hold 100ms > worth of traffic. > _______________________________________________ > freebsd-ipfw@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-ipfw > To unsubscribe, send any mail to "freebsd-ipfw-unsubscribe@freebsd.org" > > -- View this message in context: http://www.nabble.com/problem-simulating-a-LFN-tf2837638.html#a7955491 Sent from the freebsd-ipfw mailing list archive at Nabble.com.