From owner-freebsd-stable@FreeBSD.ORG Fri May 4 02:59:54 2007 Return-Path: X-Original-To: freebsd-stable@freebsd.org Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 90BDF16A400; Fri, 4 May 2007 02:59:54 +0000 (UTC) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.freebsd.org (Postfix) with ESMTP id 76B8F13C44B; Fri, 4 May 2007 02:59:54 +0000 (UTC) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.13.8/8.13.7) with ESMTP id l442xr6C078827; Thu, 3 May 2007 19:59:53 -0700 (PDT) Received: (from dillon@localhost) by apollo.backplane.com (8.13.8/8.13.4/Submit) id l442xrjd078826; Thu, 3 May 2007 19:59:53 -0700 (PDT) Date: Thu, 3 May 2007 19:59:53 -0700 (PDT) From: Matthew Dillon Message-Id: <200705040259.l442xrjd078826@apollo.backplane.com> To: "Marc G. Fournier" References: <200705040126.l441QUZh078197@apollo.backplane.com> <8B91F8463484DAC35543C340@ganymede.hub.org> Cc: Robert Watson , freebsd-stable@freebsd.org Subject: Re: Socket leak (Was: Re: What triggers "No Buffer Space) Available"? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 May 2007 02:59:54 -0000 :*groan* why couldn't this be happening on a server that I have better remote :access to? :( : :But, based on your explanation(s) above ... if I kill off all of the jail(s) on :the machine, so that there are minimal processes running, shouldn't I see a :significant drop in the number of sockets in use as well? or is there :something special about single user mode vs just killing off all 'extra :processes'? : :- ---- :Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Yes, you can. Nothing special about single user... just kill all the processes that might be using sockets. Killing the jails is a good start. If you are running a lot of jails then I would strongly suspect that there is an issue with file desciptor passing over unix domain sockets. In particular, web servers, databases, and java or other applets could be the culprit. Other possibilities... you could just be running out of file descriptors in the file descriptor table. use vmstat -m and vmstat -z too... find out what allocates the socket memory and see what it reports. Check your mbuf allocation statistics too (netstat -m). Damn, I wish that information were collected on a per-jail basis but I don't think it is. Look at all the memory statistics and check to see if anything is growing unbounded over a long period of time (verses just growing into a cache balance). Create a cron job that dumps memory statistics once a minute to a file then break each report with a clear-screen sequence and cat it in a really big xterm window. -Matt Matthew Dillon