From owner-freebsd-stable@FreeBSD.ORG Sat Sep 27 20:31:47 2008 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 69DB6106570F for ; Sat, 27 Sep 2008 20:31:47 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from QMTA05.emeryville.ca.mail.comcast.net (qmta05.emeryville.ca.mail.comcast.net [76.96.30.48]) by mx1.freebsd.org (Postfix) with ESMTP id 4EC3D8FC17 for ; Sat, 27 Sep 2008 20:31:46 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from OMTA10.emeryville.ca.mail.comcast.net ([76.96.30.28]) by QMTA05.emeryville.ca.mail.comcast.net with comcast id Kocq1a00E0cQ2SLA5wXm8j; Sat, 27 Sep 2008 20:31:46 +0000 Received: from koitsu.dyndns.org ([67.180.253.227]) by OMTA10.emeryville.ca.mail.comcast.net with comcast id KwXl1a0024v8bD78WwXlMQ; Sat, 27 Sep 2008 20:31:46 +0000 X-Authority-Analysis: v=1.0 c=1 a=QycZ5dHgAAAA:8 a=EyyiAV7YKsM3hZp1AOEA:9 a=hAig4xVbVjInWf7ClX_Z5-ELBYEA:4 a=EoioJ0NPDVgA:10 a=LY0hPdMaydYA:10 Received: by icarus.home.lan (Postfix, from userid 1000) id D365EC9432; Sat, 27 Sep 2008 13:31:44 -0700 (PDT) Date: Sat, 27 Sep 2008 13:31:44 -0700 From: Jeremy Chadwick To: Miroslav Lachman <000.fbsd@quip.cz> Message-ID: <20080927203144.GB60980@icarus.home.lan> References: <98425339-23F8-4A90-8CF1-2E85DD82D857@ish.com.au> <20080927030204.GB40195@icarus.home.lan> <48DE9411.8010002@quip.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <48DE9411.8010002@quip.cz> User-Agent: Mutt/1.5.18 (2008-05-17) Cc: freebsd-stable Stable Subject: Re: sysctl maxfiles X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Sep 2008 20:31:47 -0000 On Sat, Sep 27, 2008 at 10:14:09PM +0200, Miroslav Lachman wrote: > Jeremy Chadwick wrote: >> On Sat, Sep 27, 2008 at 11:10:01AM +1000, Aristedes Maniatis wrote: >> >>> By default FreeBSD 7.0 shipped with the sysctls set to: >>> >>> kern.maxfiles: 12328 >>> kern.maxfilesperproc: 11095 > > [...] > >> Anyway, I'd like to know why you have so many fds open simultaneously in >> the first place. We're talking over 11,000 fds actively open at once -- >> this is not a small number. What exactly is this machine doing? Are >> you absolutely certain tuning this higher is justified? Have you looked >> into the possibility that you have a program which is exhausting fds by >> not closing them when finished? (Yes, this is quite common; I've seen >> bad Java code cause this problem on Solaris.) > > I can imagine some webhosting machine running Apache virtualhosts. Each > virtual host using 3 logfiles (access log, error log, IO log) so it is > "only" about 4000 domains (virtualhosts) which is not so uncommon in > these days ;) We're a web/shell hosting provider who used to do it that way. It became unreasonable/impossible to manage. Also, if said logfiles are being placed in directories where users of those virtualhosts can remove the files (and make symlinks to other places), that's a security hole (because Apache opens webserver logfiles as root). The way we do it is much more resource-friendly: log everything to a single logfile, then every night split the logfile up (based on the CustomLog %v parameter into per-vhost log files. Apache comes with a script to do this called split-logfile. > I don't know what files are "really" open in the meaning of > kern.maxfiles. I have webserver with about 100 hosted domains and there > is some numbers: > > root@roxy ~/# fstat -u www | wc -l > 9931 I don't think this is an accurate portrait of the number of open files. The number is going to be too high; I believe entries that contain FD=jail/mmap/root/text/tr/wd are not actual descriptors (are they?) > root@roxy ~/# fstat -u root | wc -l > 718 > root@roxy ~/# fstat | grep httpd | wc -l > 6379 > root@roxy ~/# fstat | grep httpd | wc -l > 6002 > root@roxy ~/# fstat -u www | wc -l > 4691 > root@roxy ~/# sysctl kern.openfiles > kern.openfiles: 846 > > All above taken within few seconds. > > Can somebody explain the difference between kern.openfiles and fstat? -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB |