Date: Sat, 19 Jul 2008 19:28:05 -0700 From: "Kip Macy" <kip.macy@gmail.com> To: "Brian McGinty" <brian.mcginty@gmail.com> Cc: FreeBSD Net <freebsd-net@freebsd.org>, Paul <paul@gtcomm.net> Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] Message-ID: <b1fa29170807191928k74e9dfbfr156b50666e07550c@mail.gmail.com> In-Reply-To: <601bffc40807191917g131bacao6485376365304f55@mail.gmail.com> References: <4867420D.7090406@gtcomm.net> <4871E85C.8090907@freebsd.org> <48726422.7050703@gtcomm.net> <200807080107.m6817XxO021966@lava.sentex.ca> <b1fa29170807080002i1de1111fn32180ab4666402dc@mail.gmail.com> <601bffc40807081346q454c1f40td47a0f54806d8a8c@mail.gmail.com> <b1fa29170807081406o56e4211ew375c232a2cd4b682@mail.gmail.com> <601bffc40807112344n7a683f81y516f540e24d87389@mail.gmail.com> <b1fa29170807191811yaa6e515vc388693abb97aa56@mail.gmail.com> <601bffc40807191917g131bacao6485376365304f55@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sat, Jul 19, 2008 at 7:17 PM, Brian McGinty <brian.mcginty@gmail.com> wrote: > G'day Kip, > >> I'm hoping to get to it some time in August. I'm a bit behind in my >> contracts at the moment. > > A few weeks ago, I did a quick comparison of the driver between > FreeBSD and Linux, and found quite a few differences that's worth > pulling over. The guy from Intel working on FreeBSD, Jack?, is he the > one that does this sort of sync-up of the drivers between the two > distribution, or you? There's been a lot of changes recently, > including full support for multiple Rx/Tx queues that significantly > ups the ante on performance. FreeBSD doesn't support multiple Rx/Tx, > or does something half arsed. This is on a variant of RELENG_6 FreeBSD with a recent version of ULE and running the Checkpoint firewall. It also uses the full number of queues available to igb (4) and #queues == #cores (8 in this case) for ixgbe. The drivers in CVS have some bugs that I have fixed in this FreeBSD variant. FreeBSD's CVS version of the Intel drivers definitely lags Linux in terms of some optimizations. Even my version doesn't have some of the linux optimizations. >> FYI: I'm actually able to forward 2.3Mpps between 2 10Gig interfaces >> on an 8-core system. I'm hoping to push it up to 3Mpps. This is testing with an IXIA I don't currently have zero loss numbers. This is not fully loaded. However, ixgbe spews out pause frames when rx gets backed up so losses never get much above 0.1%. > Is this no-loss number, and how did you test it? I don't have > throughput numbers for the Oplin. I'm waiting to get some time on the > Ixia at work to generate performance numbers for 1G and 10G for all > packet sizes, on FreeBSD and Linux, on a 16 core system, and blast it > to the list. I expect Linux to do 2-3 times better :-) Sure, if you don't care about packet reordering. On their own box Checkpoint claims that Linux is currently able to do 20% better than we are seeing. Even they don't claim 200% - 300%. I know people who are switching off of Linux for memcache because they simply can't make it perform. So you're mileage really varies depending on the workload. I'm not sure where you get your numbers from. I would really like to get a hold of this magical Linux distribution to do a side by side comparison on the same workload. A 200% - 300% performance delta would definitely justify switching. Thanks, Kip
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?b1fa29170807191928k74e9dfbfr156b50666e07550c>