Date: Thu, 17 Jul 2008 21:45:28 +0100 From: Istvan Szukacs <leccine@gmail.com> To: freebsd-performance@freebsd.org Cc: Alexander Strange <astrange@ithinksw.com> Subject: Re: Large number of http connections immediately dropped Message-ID: <487FAF68.6040200@gmail.com> In-Reply-To: <E6D474AE-2295-4A13-8FF9-FD24404FBC80@ithinksw.com> References: <E6D474AE-2295-4A13-8FF9-FD24404FBC80@ithinksw.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi! Something to read: http://people.freebsd.org/~hmp/utilities/satbl/sysctl-net.html I have these in the sysctl.conf kern.ipc.somaxconn=4096 net.inet.tcp.recvspace=78840 net.inet.tcp.sendspace=78840 kern.ipc.shmmax=67108864 kern.ipc.shmmni=200 kern.ipc.shmseg=128 kern.ipc.semmni=70 net.local.stream.sendspace=82320 net.local.stream.recvspace=82320 net.inet.tcp.local_slowstart_flightsize=10 net.inet.tcp.nolocaltimewait=1 net.inet.tcp.hostcache.expire=3900 and the loader.conf kern.maxusers=512 kern.ipc.nmbclusters=32768 kern.ipc.maxsockets=81920 kern.ipc.maxsockbuf=1048576 net.inet.tcp.tcbhashsize=4096 net.inet.tcp.hostcache.hashsize=1024 Regards, Istvan Alexander Strange wrote: > We're running a rather high-load webserver using FreeBSD > 7-RELEASE/amd64/nginx on an Intel em gigabit connection. > Performance is good for our current bandwidth use (about 20Mbit and > ~2000 connections/sec at the moment), but a large number of HTTP > requests are being immediately dropped before getting to nginx. I see > complaints about this with earlier versions of FreeBSD - > http://forum.lighttpd.net/topic/171 - but no solutions. Does anyone > know what could be the problem, or anything we could do about it? > > There are several other servers running earlier FreeBSDs on i386 which > don't seem to have this problem, but I still haven't ruled out > upstream hardware problems or Sandvine yet. > > On the server: > -nginx's error log is full of "accept() failed (53: Software caused > connection abort)", sometimes printing three or four at the same time. > > -messages is full of: > Limiting open port RST response from 441 to 200 packets/sec > Limiting open port RST response from 488 to 200 packets/sec > Limiting open port RST response from 399 to 200 packets/sec > Limiting open port RST response from 434 to 200 packets/sec > Limiting open port RST response from 308 to 200 packets/sec > I'm not sure if that's related or not. > > -sysctl.conf: > > net.inet.tcp.tso=1 > kern.ipc.somaxconn=10240 > kern.ipc.nmbclusters=65536 > net.inet.tcp.sendspace=65536 > net.inet.tcp.recvspace=65536 > net.inet.tcp.rfc1323=1 > kern.ipc.maxsockbuf=262144 > net.inet.tcp.blackhole=2 > net.inet.udp.blackhole=1 > net.inet.tcp.msl=7500 > net.inet.icmp.icmplim=400 > net.inet.tcp.drop_synfin=1 > net.inet.tcp.icmp_may_rst=0 > net.inet.tcp.fast_finwait2_recycle=1 > > -netstat -m: > 4677/6603/11280 mbufs in use (current/cache/total) > 1017/2643/3660/65536 mbuf clusters in use (current/cache/total/max) > 1017/1961 mbuf+clusters out of packet secondary zone in use > (current/cache) > 9/514/523/12800 4k (page size) jumbo clusters in use > (current/cache/total/max) > 0/0/0/6400 9k jumbo clusters in use (current/cache/total/max) > 0/0/0/3200 16k jumbo clusters in use (current/cache/total/max) > 3239K/8992K/12232K bytes allocated to network (current/cache/total) > 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for jumbo clusters denied (4k/9k/16k) > 0/0/0 sfbufs in use (current/peak/max) > 0 requests for sfbufs denied > 0 requests for sfbufs delayed > 9204 requests for I/O initiated by sendfile > 0 calls to protocol drain routines > > nginx is not running any accept filters. > > Locally, after sending an HTTP request, I get a normal connection > close, then one RST with sequence 1, then another (possibly more than > one) RST with sequence 2. I can post a tcpdump sequence if necessary, > after I sanitize some cookies away. > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to > "freebsd-performance-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?487FAF68.6040200>