From owner-freebsd-net@FreeBSD.ORG Wed Mar 9 21:51:08 2005 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 491E716A4CF for ; Wed, 9 Mar 2005 21:51:08 +0000 (GMT) Received: from outgoing.redshift.com (outgoing.redshift.com [207.177.231.8]) by mx1.FreeBSD.org (Postfix) with ESMTP id 853AE43D48 for ; Wed, 9 Mar 2005 21:51:07 +0000 (GMT) (envelope-from ray@redshift.com) Received: from workstation (216-228-19-21.dsl.redshift.com [216.228.19.21]) by outgoing.redshift.com (Postfix) with SMTP id 8823697081 for ; Wed, 9 Mar 2005 13:51:04 -0800 (PST) Message-Id: <3.0.1.32.20050309135120.00a7f5c0@pop.redshift.com> X-Mailer: na X-Sender: redshift.com Date: Wed, 09 Mar 2005 13:51:20 -0800 To: freebsd-net@freebsd.org From: ray@redshift.com Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Subject: FreeBSD router question X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Mar 2005 21:51:08 -0000 Hello (just signed up to this list), I am wondering if anyone on the list has any experience using FreeBSD 5.3 as a router in a high traffic environment? I am building a development cluster here and have decided to try using FreeBSD as my main network router instead of something like the Cisco 7200's, Force10, etc. I have 10 or 12 Xeon machines in my cluster so far, but may have as many as 50 to 100 in the future (once our site goes live). Right now I have a 2.40 GHz Xeon with 2GB of RAM running as the router using FreeBSD 5.3, ipf and ipnat (this may be upgraded to an AMD64 bit dual core shortly). So far everything seems to work fine, but it has not been under heavy load yet. The router has been up for 26 days with no problems and works great. I've made the following tweaks (see end of message) to sysctl.conf in an effort to get things going the right direction. I've also stripped down the kernel file and recompiled. I read recently that FreeBSD was able to route 1Mpps, which sounded pretty good, but I don't know if there are any specific tweaks I need to make in order to obtain this sort of speed, or how fast it works "out of the box" with just a few modifications? My main concern is that the router works okay now, but when traffic ramps up, it hits a wall without some large amount of exotic changes. I'd like to feel comfortable that the machine will handle at least 50 to 100 megabits of traffic on a fairly sustained basis without facing any major problems. Is that realistic or are there specific changes I should make to the OS? If anyone on the list has any first hand information/experience that might steer me the right direction, that would be great. Any feed back would be great, Thanks very much! :-) Ray /etc/sysctl.conf net.inet.ip.forwarding=1 # enable packet forwarding net.inet.ip.fastforwarding=0 # not sure about this, but might want to change to 1 net.inet.ip.check_interface=1 # verify incoming packets arrives on an interface w/ address matching the packet 's destination address net.link.ether.inet.log_arp_wrong_iface=0 # turn off ARP error messages - see http://www.freebsdhowtos.com/102.htm l net.inet.tcp.blackhole=2 # drop SYN packets destine to non-listening tcp/udp ports. This will net.inet.udp.blackhole=1 # create a blackhole and protect against stealth port scans net.inet.tcp.recvspace=65535 # increase TCP window size for better network performance net.inet.tcp.sendspace=65535 kern.ipc.somaxconn=1024 # increase listen queue (defense against SYN attacks, better performance) [128] net.inet.icmp.drop_redirect=1 # disable redirects [0] net.inet.icmp.log_redirect=1 # [0] net.inet.ip.redirect=0 # [1] # net.inet6.ip6.redirect=0 # not using IPv6 net.inet.ip.sourceroute=0 # disable source routing [0] net.inet.ip.accept_sourceroute=0 # [0] net.inet.icmp.bmcastecho=0 # disable broadcast ECHO response [0] net.inet.icmp.maskrepl=0 # disable other broadcast probes [0] net.link.ether.inet.max_age=1200 # ARP clean up time (prevent flooding ARP requests) [1200]