From owner-freebsd-questions Tue Jun 19 7:54:10 2001 Delivered-To: freebsd-questions@freebsd.org Received: from mail.communityconnect.com (mail.communityconnect.com [209.10.169.57]) by hub.freebsd.org (Postfix) with ESMTP id 9EF8037B401 for ; Tue, 19 Jun 2001 07:54:00 -0700 (PDT) (envelope-from marius@mail.communityconnect.com) Received: from amavis by mail.communityconnect.com with scanned-ok (Exim 3.20 #1) id 15CMtD-0008Vh-00 for questions@freebsd.org; Tue, 19 Jun 2001 10:53:59 -0400 Received: from [206.28.215.90] (helo=utterlux.hq.communitconnect.com) by mail.communityconnect.com with esmtp (Exim 3.20 #1) id 15CMt5-0008Ed-00; Tue, 19 Jun 2001 10:53:51 -0400 Date: Tue, 19 Jun 2001 10:53:54 -0400 (EDT) From: Marius X-Sender: marius@utterlux.hq.communitconnect.com To: jaime@snowmoon.com Cc: questions@FreeBSD.org, stable@FreeBSD.org Subject: Re: running out of buffer space for sockets Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-questions@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG *smile* Given my sometimes hideous abuse of the English language, it does not surprise me that someone would think I made a typo. Actually, I meant what I said. I am using thttpd not httpd. It is a single instance, spawning no children. You have a point though, I will indeed scan the docs to see if there is anything on thttpd's side that can be done to quell the tide. The pecular part about this is that my 3.5-Stable servers don't have such problems. With the same augents to the kernel (maxusers at 128 and NMBCLUSTERS at 16384) my 4.2-Stable box had significantly fewer kernel resources then my 3.5-Stable boxen. It would freeze up and die (couldn't open more filehandles) after about a day of serving. I manually uped kern.maxfiles, kern.maxfilesperproc, kern.maxvnodes with sysctl to give it values equal to the cluster of 3.5-Stable boxen that could serve for weeks without any such problems. (I think it helped) But now after +2 weeks up uptime I am running buffer space to create more sockets. If I can't create sockets, I can't network. Yes, the root of this problem is that I don't even know how to check how much buffer space is available for the creation of new sockets. Whether it is a kernal tweak/config or something far more basic, I am striving to understand the problem so I can resolve it. This is a bit out of my normal area of experience, and I am trying to learn all that I can. Any guidence from idividuals or list members would be appreciated. -Marius B.T.W. Please cc: me in all replies. > One of my webservers is having an on-again, off-again problem with > networking. When the load gets up there (thttpd is pushing maybe 15MB > right now) I begin to run out of buffer space to create sockets. Heya old pal. :) Honestly, I think that looking at the kernel in this case is like rebuilding the foundation of a house when all that's needed is an extra layer of paint in the basement. You know, massive over kill. Look into your apache config files for something that will help. I found the following in mine: # # MaxRequestsPerChild: the number of requests each child process is # allowed to process before the child dies. The child will exit so # as to avoid problems after prolonged use when Apache (and maybe the # libraries it uses) leak memory or other resources. On most systems, this # isn't really needed, but a few (such as Solaris) do have notable leaks # in the libraries. For these platforms, set to something like 10000 # or so; a setting of 0 means unlimited. # # NOTE: This value does not include keepalive requests after the initial # request per connection. For example, if a child process handles # an initial request and 10 subsequent "keptalive" requests, it # would only count as 1 request towards this limit. # MaxRequestsPerChild 0 You should just tell apache to "recycle" child processes after they answer a certain number of requests. This will keep them from getting too big. Good luck, Jaime ---------- Forwarded message ---------- Date: Mon, 18 Jun 2001 18:06:30 -0400 (EDT) From: Marius To: questions@FreeBSD.org Subject: running out of buffer space for sockets One of my webservers is having an on-again, off-again problem with networking. When the load gets up there (thttpd is pushing maybe 15MB right now) I begin to run out of buffer space to create sockets. >Jun 18 16:44:01 cc124 sendmail[39781]: f5IKi1439781: SYSERR(root): makeconnection: cannot create socket: No buffer space available >Jun 18 16:44:01 cc124 sendmail[39781]: f5IKi1439781: SYSERR(root): makeconnection: cannot create socket: No buffer space available I get similar or related messages when I try to ping someone, use tcpdump, do an nslookup etc. Running one of these the commands in quick succession produces some successes, some failures, right after one another. I have networking fairly beefed up: ># sysclt -a |grep max >kern.maxvnodes: 129508 >kern.maxproc: 2068 >kern.maxfiles: 65536 >...snip... >kern.maxfilesperproc: 65500 >kern.maxprocperuid: 2067 >kern.ipc.maxsockbuf: 262144 >kern.ipc.somaxconn: 256 >...snip... >kern.ipc.maxsockets: 16384 >...snip... >vfs.maxbufspace: 117932032 >vfs.maxmallocbufspace: 5863833 ># netstat -m >7115/8624/65536 mbufs in use (current/peak/max): > 5454 mbufs allocated to data > 1661 mbufs allocated to packet headers >5056/5854/16384 mbuf clusters in use (current/peak/max) >13864 Kbytes allocated to network (28% of mb_map in use) >0 requests for memory denied >0 requests for memory delayed >0 calls to protocol drain routines Basically I am trying to figure out if I can tweak something to make this scarce resource less scarce. I could try changing some of the MIB's but I am not totally sure whcih ones would effect this. I am researching this the best I can, but I feel a bit overwhelmed. I'd hate to tweak something up beyond sane levels unknowingly. Pointers or suggestions would be appreciated. system info: 4.2-STABLE FreeBSD Dual 700Mhz Processors 1 GB RAM 2 eepro100 NIC cards (100baseTX ) -Marius To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-questions" in the body of the message