From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 09:47:06 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B54401065679 for ; Mon, 7 Jul 2008 09:47:06 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 0CE598FC15 for ; Mon, 7 Jul 2008 09:47:05 +0000 (UTC) (envelope-from andre@freebsd.org) Received: (qmail 30592 invoked from network); 7 Jul 2008 08:37:34 -0000 Received: from localhost (HELO [127.0.0.1]) ([127.0.0.1]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 7 Jul 2008 08:37:34 -0000 Message-ID: <4871E618.1080500@freebsd.org> Date: Mon, 07 Jul 2008 11:47:04 +0200 From: Andre Oppermann User-Agent: Thunderbird 1.5.0.14 (Windows/20071210) MIME-Version: 1.0 To: Paul References: <4867420D.7090406@gtcomm.net> <20080701033117.GH83626@cdnetworks.co.kr><4869ACFC.5020205@gtcomm.net> <4869B025.9080006@gtcomm.net><486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net><486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net><486B4F11.6040906@gtcomm.net><486BC7F5.5070604@gtcomm.net><20080703160540.W6369@delplex.bde.org><486C7F93.7010308@gtcomm.net><20080703195521.O6973@delplex.bde.org><486D35A0.4000302@gtcomm.net><486DF1A3.9000409@gtcomm.net><486E65E6.3060301@gtcomm.net> <2d3001c8def1$f4309b90$020b000a@bartwrkstxp> <486FFF70.3090402@gtcomm.net> <48701921.7090107@gtcomm.net> In-Reply-To: <48701921.7090107@gtcomm.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Net , Bart Van Kerckhove , Ingo Flaschberger Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2008 09:47:06 -0000 Paul, to get a systematic analysis of the performance please do the following tests and put them into a table for easy comparison: 1. inbound pps w/o loss with interface in monitor mode (ifconfig em0 monitor) 2. inbound pps w/ fastforward into a single blackhole route 3. inbound pps /w fastforward into a single blackhole route w/ ipfw and just one allow all rule 4. inbound pps /w fastforward into a single blackhole route w/ ipfw and just one deny all rule 5. inbound pps /w fastforward into the disc(4) discard network interface 6. inbound pps /w fastforward into the disc(4) discard network interface w/ ipfw and just one allow all rule All surrounding parameters like RX/TX interface queue length, scheduler and so may me varied but should be noted. -- Andre Paul wrote: > UP 32 bit test vs 64 bit: > negligible difference in forwarding performance without polling > slightly better polling performance but still errors at lower packet rates > same massive hit with ipfw loaded > > Installing dragonfly in a bit.. > If anyone has a really fast PPC type system or SUN or something i'd love > to try it :) > Something with a really big L1 cache :P > > > Paul wrote: >> ULE + PREEMPTION for non SMP >> no major differences with SMP with ULE/4BSD and preemption ON/OFF >> >> 32 bit UP test coming up with new cpu >> and I'm installing dragonfly sometime this weekend :] >> UP: 1mpps in one direction with no firewall/no routing table is not >> too bad, but 1mpps both directions is the goal here >> 700kpps with full bgp table in one direction is not too bad >> Ipfw needs a lot of work, barely gets 500kpps with no routing table >> with a few ipfw rules loaded.. that's horrible >> Linux barely takes a hit when you start loading iptables rules , but >> then again linux has a HUGE problem with routing >> random packet sources/ports .. grr >> My problem Is I need some box to do fast routing and some to do >> firewall.. :/ >> I'll have 32 bit 7-stable UP test with ipfw/routing table and then >> move on to dragonfly. >> I'll post the dragonfly results here as well as sign up for their >> mailing list. >> >> >> Bart Van Kerckhove wrote: >>> -----BEGIN PGP SIGNED MESSAGE----- >>> Hash: SHA1 >>> >>> Paul / Ingo, >>> >>>>> I tried all of this :/ still, 256/512 descriptors seem to work the >>>>> best. Happy to let you log into the machine and fiddle around if you >>>>> want :) >>> I've been watching this thread closely, since I'm in a very similair >>> situation. >>> A few questions/remarks: >>> >>> Does ULE provide better performance than 4BSD for forwarding? >>> Did you try freebsd4 as well? This thread had a report about that quite >>> opposite to my own experiences, -4 seemed to be a lot faster at >>> forwarding >>> than anything else I 've tried so far. >>> Obviously the thing I'm interested in is IMIX - and 64byte packets. >>> Does anyone have any benchmarks for DragonFly? I asked around on IRC, >>> but >>> that nor google turned up any useful results. >>> >>> >>>> I don't think you will be able to route 64byte packets at 1gbit >>>> wirespeed (2Mpps) with a current x86 platform. >>>> >>> Are there actual hardware related reasons this should not be >>> possible, or >>> is this purely lack of dedicated work towards this goal? >>> >>> >>> >>>> Theres a "sun" used at quagga dev as bgp-route-server. >>>> http://quagga.net/route-server.php >>>> (but they don't answered my question regarding fw-performance). >>>> >>> >>> >>> the Quagga guys are running a sun T1000 (niagara 1) route server - I >>> happen >>> to have the machine in my racks, >>> please let me know if you want to run some tests on it, I'm sure they >>> won't >>> mind ;-) >>> It should also make a great testbed for SMP performance testing imho >>> (and >>> they're pretty cheap these days) >>> Also, feel free to use me as a relay for your questions, they're not >>> always >>> very reachable. >>> >>> >>> >>>> Perhaps you have some better luck at some different hardware systems >>>> (ppc, mips, ..?) or use freebsd only for routing-table-updates and >>>> special network-cards (netfpga) for real routing. >>>> >>> The netfpga site seems more or less dead - is this project still alive? >>> It does look like a very interesting idea, even though it's currently >>> quite >>> linux-centric (and according to docs doesn't have VLAN nor ip6 >>> support, the >>> former being quite a dealbreaker) >>> >>> Paul: I'm looking forward to the C2D 32bit benchmarks (maybe throw in a >>> freebsd4 and/or dragonfly bench if you can..) - appreciate the lots of >>> information you are providing us :) >>> >>> Met vriendelijke groet / With kind regards, >>> >>> Bart Van Kerckhove >>> http://friet.net/pgp.txt >>> >>> -----BEGIN PGP SIGNATURE----- >>> >>> iQA/AwUBSG/tMgoIFchBM0BKEQKUSQCcCJqsw2wtUX7HQi050HEDYX3WPuMAnjmi >>> eca31f7WQ/oXq9tJ8TEDN3CA >>> =YGYq >>> -----END PGP SIGNATURE----- >>> >>> >>> >> >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > >