From owner-freebsd-net@FreeBSD.ORG Thu Jan 3 09:48:05 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 55B5A16A421 for ; Thu, 3 Jan 2008 09:48:05 +0000 (UTC) (envelope-from fazaeli@sepehrs.com) Received: from sepehrs.com (sepehrs.com [213.217.59.98]) by mx1.freebsd.org (Postfix) with ESMTP id 89F9113C45D for ; Thu, 3 Jan 2008 09:48:04 +0000 (UTC) (envelope-from fazaeli@sepehrs.com) Received: from [192.168.1.180] ([192.168.1.180]) by sepehrs.com (8.13.6/8.13.6) with ESMTP id m03D9TK6008581; Thu, 3 Jan 2008 13:09:30 GMT (envelope-from fazaeli@sepehrs.com) Message-ID: <477CAEB5.5070305@sepehrs.com> Date: Thu, 03 Jan 2008 13:15:25 +0330 From: "H.fazaeli" User-Agent: Thunderbird 2.0.0.9 (Windows/20071031) MIME-Version: 1.0 To: Tiffany Snyder References: <43B45EEF.6060800@x-trader.de> <43B47CB5.3C0F1632@freebsd.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Sepehr-MailScanner-Information: Please contact the ISP for more information X-Sepehr-MailScanner: Found to be clean X-Sepehr-MailScanner-SpamCheck: not spam, SpamAssassin (not cached, score=-3.921, required 5, autolearn=not spam, ALL_TRUSTED -1.80, BAYES_00 -2.60, DATE_IN_PAST_03_06 0.48) X-MailScanner-From: fazaeli@sepehrs.com X-Spam-Status: No Cc: freebsd-net@freebsd.org Subject: Re: Routing SMP benefit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Jan 2008 09:48:05 -0000 Hi all Where are 'those numbers' you are referring to? have I missed some message in the thread? Tiffany Snyder wrote: > Hi Andre, > are those numbers for small (64 bytes) packets? Good job on pushing > the base numbers higher on the same HW. > > What piqued my attention was the note that our forwarding > performance doesn't scale with multiple CPUs. Which means there's a lot of > work to be done :-) Have we taken a look at OpenSolaris' Surya > (http://www.opensolaris.org/os/community/networking/surya-design.pdf) > project? They allow multiple readers/single writer on the radix_node_head > (and not a mutex as we do) and we may be able to do the same to gain some > parallelism. There are other things in Surya that exploit multiple CPUs. > It's definitely worth a read. DragonFlyBSD seems to achieve parallelism by > classifying packet as flows and then redirecting the flows to different > CPUs. OpenSolaris also does something similar. We can definitely think along > those lines. > > NOTE: > 1) I said multiple instead of dual CPUs on purpose. > 2) I mentioned OpenSolaris and DragonFlyBSD as examples and to acknowledge > the work they are doing and to show that FreeBSD is far behind and is losing > it's lustre on continuing to be the networking platform of choice. > > Thanks, > > Tiffany. > > > On 12/29/05, Andre Oppermann wrote: > > >> Markus Oestreicher wrote: >> >>> Currently running a few routers on 5-STABLE I have read the >>> recent changes in the network stack with interest. >>> >> You should run 6.0R. It contains many improvements over 5-STABLE. >> >> >>> A few questions come to my mind: >>> >>> - Can a machine that mainly routes packets between two em(4) >>> interfaces benefit from a second CPU and SMP kernel? Can both >>> CPUs process packets from the same interface in parallel? >>> >> My testing has shown that a machine can benefit from it but not >> much in the forwarding performance. The main benefit is the >> prevention of lifelock if you have very high packet loads. The >> second CPU on SMP keeps on doing all userland tasks and running >> routing protocols. Otherwise your BGP sessions or OSPF hellos >> would stop and remove you from the routing cloud. >> >> >>> - From reading the lists it appears that net.isr.direct >>> and net.ip.fastforwarding are doing similar things. Should >>> they be used together or rather not? >>> >> net.inet.ip.fastforwarding has precedence over net.isr.direct and >> enabling both at the same doesn't gain you anything. Fastforwarding >> is about 30% faster than all other methods available, including >> polling. On my test machine with two em(4) and an AMD Opteron 852 >> (2.6GHz) I can route 580'000 pps with zero packet loss on -CURRENT. >> An upcoming optimization that will go into -CURRENT in the next >> few days pushes that to 714'000 pps. Futher optimizations are >> underway to make a stock kernel do close to or above 1'000'000 pps >> on the same hardware. >> >> -- >> Andre >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to " freebsd-net-unsubscribe@freebsd.org" >> >> > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > > > -- With best regards. Hooman Fazaeli Sepehr S. T. Co. Ltd.