Date: Tue, 19 Apr 2005 14:18:36 -0700 From: Michael DeMan <michael@staff.openaccess.org> To: Eivind Hestnes <eivind@stabbursmoen.no> Cc: performance@freebsd.org Subject: Re: Performance Intel Pro 1000 MT (PWLA8490MT) Message-ID: <a7e9487d9e1610bd23ad2b8b529b4dec@staff.openaccess.org> In-Reply-To: <426573D9.8080607@stabbursmoen.no> References: <20050419183335.F18008131@joshua.stabbursmoen.no> <5c05f1805041911351d2bd98e@mail.gmail.com> <42656CA0.9040403@stabbursmoen.no> <d00578af6eff12d7aa5139bbb410a1d2@staff.openaccess.org> <426573D9.8080607@stabbursmoen.no>
next in thread | previous in thread | raw e-mail | index | archive | help
Yes, Its also important to differentiate between routing and switching needs. Not in the regular layer-3 and layer-2 concept, but in the deployment environment you anticipate. If you really need high throughput ports, nothing will beat a regular switch (layer-2 or layer-3) because Cisco, 3COM, etc, all have dedicated hardware that will give you wire-speed gigabit, and nearly a gigabit at layer-3. Although they use slow CPUs, the bulk of the work is done in the hardware of the ports/blades themselves, not the CPU. On the other hand, with a UNIX-like router, the bulk of the packet processing is done in the CPU, and hence the need for far more CPU power to keep up with the network interfaces. In our case, being an ISP, we actually don't need much on the main backbone links and in many cases use Soekris boxes that are 133MHz and give us 30-40Mbit throughput. However, in office environments, or connecting branch offices with fiber, you may very well want that full gigabit of speed and the appropriate solution is a real switch, not a UNIX-like router. One advantage of the UNIX-like solution is in handling routing tables and such. Since Cisco/3COM/etc typically use slow CPUs and have low limits on the amount of RAM they can take, if you have large routing tables for OSPF or BGP or something, the UNIX-like machines are far cheaper to deploy. Anyway, just my 2-cents from right now. I'm always learning more about this all the time. - mike Michael F. DeMan Director of Technology OpenAccess Network Services Bellingham, WA 98225 michael@staff.openaccess.org 360-647-0785 On Apr 19, 2005, at 2:10 PM, Eivind Hestnes wrote: > It sounds sensible, but I have also learned that throwing hardware on > a problem is not always right.. Compared to shiny boxes from Cisco, HP > etc. a 500 Mhz router is for heavy duty networks. I would try some > more tweaking before replacing the box with some more spectular > hardware. > > - E. > > Michael DeMan wrote: > >> The rule of thumb I have seen on Intel/UNIX based routers is that you >> want 1GHz of CPU for every gigabit of throughput. >> >> Also, on gigabit NICs, make sure you have a 64-bit PCI bus on the >> motherboard. >> >> >> >> Michael F. DeMan >> Director of Technology >> OpenAccess Network Services >> Bellingham, WA 98225 >> michael@staff.openaccess.org >> 360-647-0785 >> On Apr 19, 2005, at 1:40 PM, Eivind Hestnes wrote: >> >>> Thanks for the advice. Didn't do any difference, though.. Perhaps I >>> should try to increase the polling frequency.. >>> >>> Jerald Von Dipple wrote: >>> >>>> Hey man >>>> >>>> You need to bump >>>> >>>> kern.polling.burst: 150 >>>> >>>> Upto at least 150000 >>>> >>>> Regards, >>>> Jerald Von D. >>>> >>>> On 4/19/05, Eivind Hestnes <eivind@stabbursmoen.no> wrote: >>>> >>>>> Hi, >>>>> >>>>> I have an Intel Pro 1000 MT (PWLA8490MT) NIC (em(4) driver 1.7.35) >>>>> installed >>>>> in a Pentium III 500 Mhz with 512 MB RAM (100 Mhz) running FreeBSD >>>>> 5.4-RC3. >>>>> The machine is routing traffic between multiple VLANs. Recently I >>>>> did a >>>>> benchmark with/without device polling enabled. Without device >>>>> polling I was >>>>> able to transfer roughly 180 Mbit/s. The router however was >>>>> suffering when >>>>> doing this benchmark. Interrupt load was peaking 100% - overall >>>>> the system >>>>> itself was quite unusable (_very_ high system load). With device >>>>> polling >>>>> enabled the interrupt kept stable around 40-50% and max transfer >>>>> rate was >>>>> nearly 70 Mbit/s. Not very scientific tests, but it gave me a pin >>>>> point. >>>>> >>>>> However, a Pentium III in combination with a good NIC should in my >>>>> opinion >>>>> be a respectful router.. but I'm not satisfied with the results. >>>>> The pf >>>>> ruleset is like nothing, and the kernel is stripped and customized >>>>> for best >>>>> performance. >>>>> >>>>> Any tweaking tips for making my router perform better? >>>>> >>>>> Debug information: >>>>> eivind@core-gw:~$ sysctl -a | grep kern.polling >>>>> kern.polling.burst: 150 >>>>> kern.polling.each_burst: 5 >>>>> kern.polling.burst_max: 150 >>>>> kern.polling.idle_poll: 0 >>>>> kern.polling.poll_in_trap: 0 >>>>> kern.polling.user_frac: 50 >>>>> kern.polling.reg_frac: 20 >>>>> kern.polling.short_ticks: 1411 >>>>> kern.polling.lost_polls: 720 >>>>> kern.polling.pending_polls: 0 >>>>> kern.polling.residual_burst: 0 >>>>> kern.polling.handlers: 0 >>>>> kern.polling.enable: 1 >>>>> kern.polling.phase: 0 >>>>> kern.polling.suspect: 186 >>>>> kern.polling.stalled: 0 >>>>> kern.polling.idlepoll_sleeping: 1 >>>>> >>>>> eivind@core-gw:~$ cat /etc/sysctl.conf >>>>> net.inet.ip.forwarding=1 >>>>> net.inet.ip.fastforwarding=1 >>>>> net.inet.carp.preempt=1 >>>>> kern.polling.enable=1 >>>>> >>>>> HZ set to 1000 as recommended in README for the em(4) driver. >>>>> Driver is of >>>>> cource compiled into kernel. >>>>> >>>>> Regards, >>>>> Eivind Hestnes >>>>> >>>>> _______________________________________________ >>>>> freebsd-performance@freebsd.org mailing list >>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-performance >>>>> To unsubscribe, send any mail to >>>>> "freebsd-performance-unsubscribe@freebsd.org" >>>>> >>>>> >>> >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to >>> "freebsd-net-unsubscribe@freebsd.org" >>> >> >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?a7e9487d9e1610bd23ad2b8b529b4dec>