Date: Tue, 04 Mar 2003 11:52:58 +0900 (KST) From: CHOI Junho <cjh@kr.FreeBSD.org> To: bmilekic@unixdaemons.com Cc: net@freebsd.org Subject: Re: Performance tuning hints of gigabit networking? Message-ID: <20030304.115258.21844180.cjh@kr.FreeBSD.org> In-Reply-To: <20030303180014.A7317@unixdaemons.com> References: <20030226.220551.10329540.cjh@kr.FreeBSD.org> <20030303180014.A7317@unixdaemons.com>
next in thread | previous in thread | raw e-mail | index | archive | help
That's 2GB machine. How much RAM I need more? mbuf clusters is full(observed by netstat -m) at peak time(3-4 hours). netstat -m output below is when the connection is low, but please see the peak value of mbuf clusters. From: Bosko Milekic <bmilekic@unixdaemons.com> Subject: Re: Performance tuning hints of gigabit networking? Date: Mon, 3 Mar 2003 18:00:14 -0500 > > > You're not running out of mbufs or clusters, you're out of RAM. > Don't bump up nmbclusters anymore because you don't need to; instead, > add more RAM. > > On Wed, Feb 26, 2003 at 10:05:51PM +0900, CHOI Junho wrote: > > > > Hi, > > > > I am looking for a good resource for kernel tuning on very high > > bandwidth HTTP servers(avg 500Mbit/sec, peak 950Mbit/sec). Today I > > faced very unusual situation with 950Mbit/sec bandwidth! > > > > > netstat -m > > 16962/93488/262144 mbufs in use (current/peak/max): > > 16962 mbufs allocated to data > > 16952/65536/65536 mbuf clusters in use (current/peak/max) > > 154444 Kbytes allocated to network (14% of mb_map in use) > > 512627 requests for memory denied > > 2614 requests for memory delayed > > 0 calls to protocol drain routines > > > > I set kern.ipc.nmbclusters=65536, but it overflowed. This is P-IV Xeon > > 1.8G, 2GB RAM, and one Intel 1000baseSX(em driver) machine running > > 4.7-RELEASE-pX. This server is running only one service, HTTP. I use > > thttpd, since apache doesn't work in such a high load. thttpd is highly > > amazing, just give <1 load in any time. > > > > Once I tried to increase kern.ipc.nmbclusters to 131072 or > > higher(multiple of 65536 or 32768, tuning(7) only cites about 32768 > > case..), it fails to boot kernel when 262144, or kernel panic in > > somewhat higher load when 131072, so I gave up other changes and fall > > back to 65536. > > > > What is a good way to calcurate this value safely? Here is another > > hint, /etc/sysctl.conf: > > > > net.link.ether.inet.log_arp_wrong_iface=0 > > kern.ipc.maxsockbuf=2048000 > > kern.ipc.somaxconn=4096 > > kern.ipc.maxsockets=60000 > > kern.maxfiles=65536 > > kern.maxfilesperproc=32768 > > net.inet.tcp.rfc1323=1 > > net.inet.tcp.delayed_ack=0 > > net.inet.tcp.sendspace=65535 > > net.inet.tcp.recvspace=65535 > > net.inet.udp.recvspace=65535 > > net.inet.udp.maxdgram=57344 > > net.inet.icmp.drop_redirect=1 > > net.inet.icmp.log_redirect=1 > > net.inet.ip.redirect=0 > > net.inet6.ip6.redirect=0 > > net.link.ether.inet.max_age=1200 > > net.inet.ip.sourceroute=0 > > net.inet.ip.accept_sourceroute=0 > > net.inet.icmp.bmcastecho=0 > > net.inet.icmp.maskrepl=0 > > net.inet.tcp.inflight_enable=1 > > > > kernel configuration is not specially tuned, except DEVICE_POLLING and > > HZ=2000. > > > > -- > > CHOI Junho <http://www.kr.FreeBSD.org/~cjh>; KFUG <cjh at kr.FreeBSD.org> > > FreeBSD Project <cjh at FreeBSD.org> Web Data Bank <cjh at wdb.co.kr> > > Key fingerprint = 1369 7374 A45F F41A F3C0 07E3 4A01 C020 E602 60F5 > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > > with "unsubscribe freebsd-net" in the body of the message > > > > -- > Bosko Milekic * bmilekic@unixdaemons.com * bmilekic@FreeBSD.org -- CHOI Junho <http://www.kr.FreeBSD.org/~cjh>; KFUG <cjh at kr.FreeBSD.org> FreeBSD Project <cjh at FreeBSD.org> Web Data Bank <cjh at wdb.co.kr> Key fingerprint = 1369 7374 A45F F41A F3C0 07E3 4A01 C020 E602 60F5 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-net" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20030304.115258.21844180.cjh>