From owner-freebsd-performance@FreeBSD.ORG Tue Apr 19 21:18:38 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4C31316A4CE for ; Tue, 19 Apr 2005 21:18:38 +0000 (GMT) Received: from smtp.openaccess.org (smtp.openaccess.org [216.57.214.76]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9E26C43D53 for ; Tue, 19 Apr 2005 21:18:37 +0000 (GMT) (envelope-from michael@staff.openaccess.org) Received: from [216.57.214.90] (unknown [216.57.214.90]) by smtp.openaccess.org (Postfix) with ESMTP id F2005416B; Tue, 19 Apr 2005 14:18:27 -0700 (PDT) In-Reply-To: <426573D9.8080607@stabbursmoen.no> References: <20050419183335.F18008131@joshua.stabbursmoen.no> <5c05f1805041911351d2bd98e@mail.gmail.com> <42656CA0.9040403@stabbursmoen.no> <426573D9.8080607@stabbursmoen.no> Mime-Version: 1.0 (Apple Message framework v619.2) Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Michael DeMan Date: Tue, 19 Apr 2005 14:18:36 -0700 To: Eivind Hestnes X-Mailer: Apple Mail (2.619.2) X-Mailman-Approved-At: Wed, 20 Apr 2005 12:22:22 +0000 cc: Jerald Von Dipple cc: performance@freebsd.org Subject: Re: Performance Intel Pro 1000 MT (PWLA8490MT) X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Apr 2005 21:18:38 -0000 Yes, Its also important to differentiate between routing and switching needs. Not in the regular layer-3 and layer-2 concept, but in the deployment environment you anticipate. If you really need high throughput ports, nothing will beat a regular switch (layer-2 or layer-3) because Cisco, 3COM, etc, all have dedicated hardware that will give you wire-speed gigabit, and nearly a gigabit at layer-3. Although they use slow CPUs, the bulk of the work is done in the hardware of the ports/blades themselves, not the CPU. On the other hand, with a UNIX-like router, the bulk of the packet processing is done in the CPU, and hence the need for far more CPU power to keep up with the network interfaces. In our case, being an ISP, we actually don't need much on the main backbone links and in many cases use Soekris boxes that are 133MHz and give us 30-40Mbit throughput. However, in office environments, or connecting branch offices with fiber, you may very well want that full gigabit of speed and the appropriate solution is a real switch, not a UNIX-like router. One advantage of the UNIX-like solution is in handling routing tables and such. Since Cisco/3COM/etc typically use slow CPUs and have low limits on the amount of RAM they can take, if you have large routing tables for OSPF or BGP or something, the UNIX-like machines are far cheaper to deploy. Anyway, just my 2-cents from right now. I'm always learning more about this all the time. - mike Michael F. DeMan Director of Technology OpenAccess Network Services Bellingham, WA 98225 michael@staff.openaccess.org 360-647-0785 On Apr 19, 2005, at 2:10 PM, Eivind Hestnes wrote: > It sounds sensible, but I have also learned that throwing hardware on > a problem is not always right.. Compared to shiny boxes from Cisco, HP > etc. a 500 Mhz router is for heavy duty networks. I would try some > more tweaking before replacing the box with some more spectular > hardware. > > - E. > > Michael DeMan wrote: > >> The rule of thumb I have seen on Intel/UNIX based routers is that you >> want 1GHz of CPU for every gigabit of throughput. >> >> Also, on gigabit NICs, make sure you have a 64-bit PCI bus on the >> motherboard. >> >> >> >> Michael F. DeMan >> Director of Technology >> OpenAccess Network Services >> Bellingham, WA 98225 >> michael@staff.openaccess.org >> 360-647-0785 >> On Apr 19, 2005, at 1:40 PM, Eivind Hestnes wrote: >> >>> Thanks for the advice. Didn't do any difference, though.. Perhaps I >>> should try to increase the polling frequency.. >>> >>> Jerald Von Dipple wrote: >>> >>>> Hey man >>>> >>>> You need to bump >>>> >>>> kern.polling.burst: 150 >>>> >>>> Upto at least 150000 >>>> >>>> Regards, >>>> Jerald Von D. >>>> >>>> On 4/19/05, Eivind Hestnes wrote: >>>> >>>>> Hi, >>>>> >>>>> I have an Intel Pro 1000 MT (PWLA8490MT) NIC (em(4) driver 1.7.35) >>>>> installed >>>>> in a Pentium III 500 Mhz with 512 MB RAM (100 Mhz) running FreeBSD >>>>> 5.4-RC3. >>>>> The machine is routing traffic between multiple VLANs. Recently I >>>>> did a >>>>> benchmark with/without device polling enabled. Without device >>>>> polling I was >>>>> able to transfer roughly 180 Mbit/s. The router however was >>>>> suffering when >>>>> doing this benchmark. Interrupt load was peaking 100% - overall >>>>> the system >>>>> itself was quite unusable (_very_ high system load). With device >>>>> polling >>>>> enabled the interrupt kept stable around 40-50% and max transfer >>>>> rate was >>>>> nearly 70 Mbit/s. Not very scientific tests, but it gave me a pin >>>>> point. >>>>> >>>>> However, a Pentium III in combination with a good NIC should in my >>>>> opinion >>>>> be a respectful router.. but I'm not satisfied with the results. >>>>> The pf >>>>> ruleset is like nothing, and the kernel is stripped and customized >>>>> for best >>>>> performance. >>>>> >>>>> Any tweaking tips for making my router perform better? >>>>> >>>>> Debug information: >>>>> eivind@core-gw:~$ sysctl -a | grep kern.polling >>>>> kern.polling.burst: 150 >>>>> kern.polling.each_burst: 5 >>>>> kern.polling.burst_max: 150 >>>>> kern.polling.idle_poll: 0 >>>>> kern.polling.poll_in_trap: 0 >>>>> kern.polling.user_frac: 50 >>>>> kern.polling.reg_frac: 20 >>>>> kern.polling.short_ticks: 1411 >>>>> kern.polling.lost_polls: 720 >>>>> kern.polling.pending_polls: 0 >>>>> kern.polling.residual_burst: 0 >>>>> kern.polling.handlers: 0 >>>>> kern.polling.enable: 1 >>>>> kern.polling.phase: 0 >>>>> kern.polling.suspect: 186 >>>>> kern.polling.stalled: 0 >>>>> kern.polling.idlepoll_sleeping: 1 >>>>> >>>>> eivind@core-gw:~$ cat /etc/sysctl.conf >>>>> net.inet.ip.forwarding=1 >>>>> net.inet.ip.fastforwarding=1 >>>>> net.inet.carp.preempt=1 >>>>> kern.polling.enable=1 >>>>> >>>>> HZ set to 1000 as recommended in README for the em(4) driver. >>>>> Driver is of >>>>> cource compiled into kernel. >>>>> >>>>> Regards, >>>>> Eivind Hestnes >>>>> >>>>> _______________________________________________ >>>>> freebsd-performance@freebsd.org mailing list >>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-performance >>>>> To unsubscribe, send any mail to >>>>> "freebsd-performance-unsubscribe@freebsd.org" >>>>> >>>>> >>> >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to >>> "freebsd-net-unsubscribe@freebsd.org" >>> >> >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > >