From owner-freebsd-stable@FreeBSD.ORG Sat Sep 27 22:18:09 2008 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C8E471065686 for ; Sat, 27 Sep 2008 22:18:09 +0000 (UTC) (envelope-from gpalmer@freebsd.org) Received: from noop.in-addr.com (mail.in-addr.com [IPv6:2001:470:8:162::1]) by mx1.freebsd.org (Postfix) with ESMTP id 886478FC20 for ; Sat, 27 Sep 2008 22:18:09 +0000 (UTC) (envelope-from gpalmer@freebsd.org) Received: from gjp by noop.in-addr.com with local (Exim 4.54 (FreeBSD)) id 1Kji75-000IPz-Pa; Sat, 27 Sep 2008 18:18:07 -0400 Date: Sat, 27 Sep 2008 18:18:07 -0400 From: Gary Palmer To: Aristedes Maniatis Message-ID: <20080927221807.GE60230@in-addr.com> References: <98425339-23F8-4A90-8CF1-2E85DD82D857@ish.com.au> <20080927030204.GB40195@icarus.home.lan> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Cc: freebsd-stable Stable Subject: Re: sysctl maxfiles X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Sep 2008 22:18:09 -0000 On Sat, Sep 27, 2008 at 07:05:08PM +1000, Aristedes Maniatis wrote: > > On 27/09/2008, at 1:02 PM, Jeremy Chadwick wrote: > > >Anyway, I'd like to know why you have so many fds open > >simultaneously in > >the first place. We're talking over 11,000 fds actively open at > >once -- > >this is not a small number. What exactly is this machine doing? Are > >you absolutely certain tuning this higher is justified? Have you > >looked > >into the possibility that you have a program which is exhausting fds > >by > >not closing them when finished? (Yes, this is quite common; I've seen > >bad Java code cause this problem on Solaris.) > > > Well, there was a runaway process which looks like it is leaking fds. > We haven't solved it yet, but the fact that the maxfiles per machine > and the maxfiles per process were so close together was really causing > us grief for a while. > > > > >You're asking for trouble setting these values to the equivalent of > >unlimited. Instead of asking "what would happen", you should be > >asking > >"why would I need to do that". > > > >Regarding memory implications, the Handbook goes over it. > > Unfortunately I've been unable to find it. While we fix the fd leak > I'd like to know how high I can push these numbers and not cause other > problems. At least one port recommends you set kern.maxfiles="40000" in /boot/loader.conf. I think its one of the GNOME ports. I'm pretty confident you can run that without too many problems, and maybe go higher, but if you really want to know the limit its probably kernel memory and that will depend on your workload. Solving the fd leak is by far the safest path. Note that tracking that many files is probably affecting your application performance in addition to hurting the system. Regards, Gary