Date: Fri, 07 Sep 2007 13:08:09 -0700 From: Bakul Shah <bakul@bitblocks.com> To: Kirc Gover <kirc.gover@yahoo.com.au> Cc: freebsd-net@freebsd.org Subject: Re: OS choice for an edge router Message-ID: <20070907200809.CB6B05B58@mail.bitblocks.com> In-Reply-To: Your message of "Sat, 08 Sep 2007 03:57:11 %2B1000." <517436.12027.qm@web44802.mail.sp1.yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
> One of my concern is on the native forwarding capability of FreeBSD OS and the > execution of critical userland processes. I have experience before that a > FreeBSD box configured as router appears to slow down the userland processes > when the traffic load is high. I have verified this lately on 6.1, running on > Athlon64 with 1G NIC cards with PF and ALTQ (queuing) enabled. I'm not so sure > if this is caused by PF or ALTQ. Looking at the processes using top, it could > see that "swi(x) net" process is almost as near 100% cpu utilization. At this > state, the box can still forward traffic (not sure yet if the was a change in > throughput) but I could notice the userland processes to be very slow, like > invoking any command from the shell (e.g. ls) will take so long to be executed > and completed. Is this a know limitation or bug? On my Athlon64 X2 3800+ box I get about 42,500 pings a second (with ping -q -f localhost, after removing icmp limits). For forwarding, you won't get a packets per second (pps) number significantly higher than that. Less if you do PF, ALTQ or VPNs. The worst case for 1GBps is about 1.488 Million pps. Even if you assume there is one ack packet for one data pacet of max size, you have over 150K pps when the full 1GBps is used up. I just don't think stock *BSD will do this for you on a typical el cheapo PC box. Neither will Linux or vmware. Listen to what Louis Mamakos said! Use FreeBSD primarily for the control plane. May be there are NICs where you can offload some packet forwarding.... But that is a substantial change to FreeBSD. Or live with what FreeBSD can do on a given box. For a project like yours, any Open Source code you get is a *starting point* and no more. Your customers don't care if it is FreeBSD or CP/M; they just expect your router to work and when it doesn't they expect you to fix it pronto. This means you have to do QA, find and fix bugs, add missing functionality and so on. You can't wait for FreeBSD volunteers to fix any problems; you will have to fix any OS panics, deadlocks and inefficiencies! May be you can start with requirements. How many VPNs, how much bandwidth, required pps, other performance/latency requirements, protocols (and specific features in them) you *must* have, protocols you may like to have, required CPU bandwidth for running your proprietary services, some idea of how these numbers will grow over the next three years, what applications you may wish to run, what cost of goods you can afford, etc. That ought to help you decide what h/w platform is suitable or which requirement must go :-)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20070907200809.CB6B05B58>