From owner-freebsd-net@FreeBSD.ORG Wed Oct 8 04:22:36 2003 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CED3216A4BF for ; Wed, 8 Oct 2003 04:22:36 -0700 (PDT) Received: from smtpzilla2.xs4all.nl (smtpzilla2.xs4all.nl [194.109.127.138]) by mx1.FreeBSD.org (Postfix) with ESMTP id BE9FD43FBF for ; Wed, 8 Oct 2003 04:22:35 -0700 (PDT) (envelope-from peter.bozarov@moniforce.com) Received: from moniforce.com (213-84-208-64.adsl.xs4all.nl [213.84.208.64]) by smtpzilla2.xs4all.nl (8.12.9/8.12.9) with ESMTP id h98BMYCT064006; Wed, 8 Oct 2003 13:22:34 +0200 (CEST) Message-ID: <3F83F37A.7020908@moniforce.com> Date: Wed, 08 Oct 2003 13:22:34 +0200 From: Peter Bozarov User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.1) Gecko/20021005 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Bruce M Simpson References: <3F83E2A7.8070209@moniforce.com> <20031008101733.GI6524@saboteur.dek.spc.org> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit cc: freebsd-net@freebsd.org Subject: Re: Recovery from mbuf cluster exhaustion X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Oct 2003 11:22:36 -0000 Bruce M Simpson wrote: > On Wed, Oct 08, 2003 at 12:10:47PM +0200, Peter Bozarov wrote: > >>(First off, I hope I'm posting to the right list.) > > > -net would be more appropriate. [Redirected from -hackers] > > >>I have the following question regarding mbuf cluster exhaustion. >>If I've managed to exhaust the pool, I start getting the usual >>"All mbuf clusters exhausted, please see tuning(7)." message. >>Now, at that point this is what my mbuf pool looked like: > > > Which version of FreeBSD are you using? > What network interfaces do you have? > What does this box do to get to this stage? > What does your routing table look like? > > BMS Hi, I'm using 4.7. I have three interfaces, rl0, xl0, and xl1. The rl0 interface is a RealTek 8139 10/100BaseTX, the xl* are two 3Com 3c905-TX Fast Etherlink XL cards. This is what my routing tables look like. Internet: Destination Gateway Flags Refs Use Netif Expire default 10.2.1.1 UGSc 4 0 rl0 10/24 link#2 UC 1 0 xl0 10.0.0.2 00:07:e9:d5:18:52 UHLW 0 4 xl0 1196 10.2.1/24 link#1 UC 2 0 rl0 10.2.1.1 00:04:76:1f:53:60 UHLW 5 58 rl0 1151 10.2.1.2 00:60:08:10:4a:36 UHLW 0 2 rl0 1179 10.2.1.68 127.0.0.1 UGHS 0 0 lo0 127.0.0.1 127.0.0.1 UH 2 30 lo0 My machine serves as a bridge between the 10.0.0.0/24 (xl0) and the 10.2.1.0/24 networks (rl0). xl1 is not being used. I've made a tcpdump of machines on the 10.2.1.0/24 network communicating with the internet. Now, on the 10.0.0.0/24, I have a sniffer. I'm trying to test the program by injecting the tcpdump file into the 10.0.0.0/24 network. Apparently, since some of the outgoing traffic on xl0 is originated on or is meant for the localhost (10.2.1.68) the xl0 driver starts buffering/passing those segments to the kernel (xl0 is in promiscuous mode). Shorty after this, I run out of mbuf clusters. If I bring the xl0 card down, my mbuf pool stops growing. The moment I bring xl0 up, I start getting the "All mbuf clusters exhausted" messages again. Bring it down, messages go away, but the clusters that xl0 is using are never released. The only solution is to reboot. How do I cause the driver (or whatever it is) that's hogging the clusters to relinquish them? Peter