Date: Mon, 07 Jul 2008 11:47:04 +0200 From: Andre Oppermann <andre@freebsd.org> To: Paul <paul@gtcomm.net> Cc: FreeBSD Net <freebsd-net@freebsd.org>, Bart Van Kerckhove <bart@it-ss.be>, Ingo Flaschberger <if@xip.at> Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] Message-ID: <4871E618.1080500@freebsd.org> In-Reply-To: <48701921.7090107@gtcomm.net> References: <4867420D.7090406@gtcomm.net> <20080701033117.GH83626@cdnetworks.co.kr><ea7b9c170806302050p2a3a5480t29923a4ac2d7c852@mail.gmail.com><4869ACFC.5020205@gtcomm.net> <4869B025.9080006@gtcomm.net><486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net><486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net><alpine.LFD.1.10.0807021052041.557@filebunker.xip.at><486B4F11.6040906@gtcomm.net><alpine.LFD.1.10.0807021155280.557@filebunker.xip.at><486BC7F5.5070604@gtcomm.net><20080703160540.W6369@delplex.bde.org><486C7F93.7010308@gtcomm.net><20080703195521.O6973@delplex.bde.org><486D35A0.4000302@gtcomm.net><alpine.LFD.1.10.0807041106591.19613@filebunker.xip.at><486DF1A3.9000409@gtcomm.net><alpine.LFD.1.10.0807041303490.20760@filebunker.xip.at><486E65E6.3060301@gtcomm.net> <alpine.LFD.1.10.0807052356130.2145@filebunker.xip.at> <2d3001c8def1$f4309b90$020b000a@bartwrkstxp> <486FFF70.3090402@gtcomm.net> <48701921.7090107@gtcomm.net>
next in thread | previous in thread | raw e-mail | index | archive | help
Paul,
to get a systematic analysis of the performance please do the following
tests and put them into a table for easy comparison:
1. inbound pps w/o loss with interface in monitor mode (ifconfig em0 monitor)
2. inbound pps w/ fastforward into a single blackhole route
3. inbound pps /w fastforward into a single blackhole route w/ ipfw and
just one allow all rule
4. inbound pps /w fastforward into a single blackhole route w/ ipfw and
just one deny all rule
5. inbound pps /w fastforward into the disc(4) discard network interface
6. inbound pps /w fastforward into the disc(4) discard network interface
w/ ipfw and just one allow all rule
All surrounding parameters like RX/TX interface queue length, scheduler
and so may me varied but should be noted.
--
Andre
Paul wrote:
> UP 32 bit test vs 64 bit:
> negligible difference in forwarding performance without polling
> slightly better polling performance but still errors at lower packet rates
> same massive hit with ipfw loaded
>
> Installing dragonfly in a bit..
> If anyone has a really fast PPC type system or SUN or something i'd love
> to try it :)
> Something with a really big L1 cache :P
>
>
> Paul wrote:
>> ULE + PREEMPTION for non SMP
>> no major differences with SMP with ULE/4BSD and preemption ON/OFF
>>
>> 32 bit UP test coming up with new cpu
>> and I'm installing dragonfly sometime this weekend :]
>> UP: 1mpps in one direction with no firewall/no routing table is not
>> too bad, but 1mpps both directions is the goal here
>> 700kpps with full bgp table in one direction is not too bad
>> Ipfw needs a lot of work, barely gets 500kpps with no routing table
>> with a few ipfw rules loaded.. that's horrible
>> Linux barely takes a hit when you start loading iptables rules , but
>> then again linux has a HUGE problem with routing
>> random packet sources/ports .. grr
>> My problem Is I need some box to do fast routing and some to do
>> firewall.. :/
>> I'll have 32 bit 7-stable UP test with ipfw/routing table and then
>> move on to dragonfly.
>> I'll post the dragonfly results here as well as sign up for their
>> mailing list.
>>
>>
>> Bart Van Kerckhove wrote:
>>> -----BEGIN PGP SIGNED MESSAGE-----
>>> Hash: SHA1
>>>
>>> Paul / Ingo,
>>>
>>>>> I tried all of this :/ still, 256/512 descriptors seem to work the
>>>>> best. Happy to let you log into the machine and fiddle around if you
>>>>> want :)
>>> I've been watching this thread closely, since I'm in a very similair
>>> situation.
>>> A few questions/remarks:
>>>
>>> Does ULE provide better performance than 4BSD for forwarding?
>>> Did you try freebsd4 as well? This thread had a report about that quite
>>> opposite to my own experiences, -4 seemed to be a lot faster at
>>> forwarding
>>> than anything else I 've tried so far.
>>> Obviously the thing I'm interested in is IMIX - and 64byte packets.
>>> Does anyone have any benchmarks for DragonFly? I asked around on IRC,
>>> but
>>> that nor google turned up any useful results.
>>>
>>> <snip>
>>>> I don't think you will be able to route 64byte packets at 1gbit
>>>> wirespeed (2Mpps) with a current x86 platform.
>>>>
>>> Are there actual hardware related reasons this should not be
>>> possible, or
>>> is this purely lack of dedicated work towards this goal?
>>>
>>> <snip>
>>>
>>>> Theres a "sun" used at quagga dev as bgp-route-server.
>>>> http://quagga.net/route-server.php
>>>> (but they don't answered my question regarding fw-performance).
>>>>
>>>
>>>
>>> the Quagga guys are running a sun T1000 (niagara 1) route server - I
>>> happen
>>> to have the machine in my racks,
>>> please let me know if you want to run some tests on it, I'm sure they
>>> won't
>>> mind ;-)
>>> It should also make a great testbed for SMP performance testing imho
>>> (and
>>> they're pretty cheap these days)
>>> Also, feel free to use me as a relay for your questions, they're not
>>> always
>>> very reachable.
>>> <snap>
>>>
>>>
>>>> Perhaps you have some better luck at some different hardware systems
>>>> (ppc, mips, ..?) or use freebsd only for routing-table-updates and
>>>> special network-cards (netfpga) for real routing.
>>>>
>>> The netfpga site seems more or less dead - is this project still alive?
>>> It does look like a very interesting idea, even though it's currently
>>> quite
>>> linux-centric (and according to docs doesn't have VLAN nor ip6
>>> support, the
>>> former being quite a dealbreaker)
>>>
>>> Paul: I'm looking forward to the C2D 32bit benchmarks (maybe throw in a
>>> freebsd4 and/or dragonfly bench if you can..) - appreciate the lots of
>>> information you are providing us :)
>>>
>>> Met vriendelijke groet / With kind regards,
>>>
>>> Bart Van Kerckhove
>>> http://friet.net/pgp.txt
>>>
>>> -----BEGIN PGP SIGNATURE-----
>>>
>>> iQA/AwUBSG/tMgoIFchBM0BKEQKUSQCcCJqsw2wtUX7HQi050HEDYX3WPuMAnjmi
>>> eca31f7WQ/oXq9tJ8TEDN3CA
>>> =YGYq
>>> -----END PGP SIGNATURE-----
>>>
>>>
>>>
>>
>> _______________________________________________
>> freebsd-net@freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-net
>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
>>
>
> _______________________________________________
> freebsd-net@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
>
>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4871E618.1080500>