From owner-freebsd-performance@FreeBSD.ORG Mon Jan 19 13:20:08 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9358E16A4CE; Mon, 19 Jan 2004 13:20:08 -0800 (PST) Received: from park.rambler.ru (park.rambler.ru [81.19.64.101]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6BB0443D54; Mon, 19 Jan 2004 13:20:01 -0800 (PST) (envelope-from is@rambler-co.ru) Received: from is (is.park.rambler.ru [81.19.64.102]) by park.rambler.ru (8.12.6/8.12.6) with ESMTP id i0JLJsAY009298; Tue, 20 Jan 2004 00:19:54 +0300 (MSK) (envelope-from is@rambler-co.ru) Date: Tue, 20 Jan 2004 00:19:54 +0300 (MSK) From: Igor Sysoev X-Sender: is@is To: CHOI Junho In-Reply-To: <20040119.192257.34695172.cjh@kr.FreeBSD.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Mailman-Approved-At: Fri, 23 Jan 2004 00:09:07 -0800 cc: freebsd-net@freebsd.org cc: freebsd-performance@freebsd.org Subject: Re: mbuf tuning X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Jan 2004 21:20:08 -0000 On Mon, 19 Jan 2004, CHOI Junho wrote: > From: Mike Silbersack > Subject: Re: mbuf tuning > Date: Mon, 19 Jan 2004 01:12:08 -0600 (CST) > > > There are no good guidelines other than "don't set it too high." Andre > > and I have talked about some ideas on how to make mbuf usage more dynamic, > > I think that he has something in the works. But at present, once you hit > > the wall, that's it. > > > > One way to reduce mbuf cluster usage is to use sendfile where possible. > > Data sent via sendfile does not use mbuf clusters, and is more memory > > efficient. If you run 5.2 or above, it's *much* more memory efficient, > > due to change Alan Cox recently made. Apache 2 will use sendfile by > > default, so if you're running apache 1, that may be one reason for an > > upgrade. > > I am using custom version of thttpd. It allocates mmap() first(builtin > method of thttpd), and it try to use sendfile() if mmap() fails(out of > mmap memory). It really works good in normal status but the problem is > that sendfile buffer is also easy to flood. I need more sendfile > buffers but I don't know how to increase sendfile buffers either(I > think it's hidden sysctl but it was more difficult to tune than > nmbclusters). With higher traffic, thttpd sometimes stuck at "sfbufa" > status when I run top(I guess it's "sendfile buffer allocation" > status). In 4.x you have to rebuild the kernel with options NSFBUFS=16384 It equals to (512 + maxusers * 16) by default. By the way, why do you want to use the big net.inet.tcp.sendspace and net.inet.tcp.recvspace ? It makes a sense for Apache but thttpd can easy work with the small buffers, say, 16K or even 8K. > > > Increasing kern.ipc.nmbclusters caused frequent kernel panic > > > under 4.7/4.8/4.9. How can I set more nmbclusters value with 64K tcp > > > buffers? Or is any dependency for mbufclusters value? (e.g. RAM size, > > > kern.maxusers value or etc) > > > > > > p.s. RAM is 2G, Xeon 2.0G x 1 or 2 machines. > > > > You probably need to bump up KVA_PAGES to fit in all the extra mbuf > > clusters you're allocating. > > Can you tell me in more detail? >From LINT: --- # # Change the size of the kernel virtual address space. Due to # constraints in loader(8) on i386, this must be a multiple of 4. # 256 = 1 GB of kernel address space. Increasing this also causes # a reduction of the address space in user processes. 512 splits # the 4GB cpu address space in half (2GB user, 2GB kernel). # options KVA_PAGES=260 --- Default KVA_PAGES are 256. Igor Sysoev http://sysoev.ru/en/