From owner-freebsd-stable@FreeBSD.ORG Wed Apr 26 13:30:48 2006 Return-Path: X-Original-To: stable@freebsd.org Delivered-To: freebsd-stable@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 893D116A406 for ; Wed, 26 Apr 2006 13:30:48 +0000 (UTC) (envelope-from Stephen.Clark@seclark.us) Received: from smtpout04-04.prod.mesa1.secureserver.net (smtpout04-04.prod.mesa1.secureserver.net [64.202.165.199]) by mx1.FreeBSD.org (Postfix) with SMTP id 2C72143D5E for ; Wed, 26 Apr 2006 13:30:40 +0000 (GMT) (envelope-from Stephen.Clark@seclark.us) Received: (qmail 32445 invoked from network); 26 Apr 2006 13:30:39 -0000 Received: from unknown (24.144.77.138) by smtpout04-04.prod.mesa1.secureserver.net (64.202.165.199) with ESMTP; 26 Apr 2006 13:30:39 -0000 Message-ID: <444F75FE.3010101@seclark.us> Date: Wed, 26 Apr 2006 09:30:38 -0400 From: Stephen Clark User-Agent: Mozilla/5.0 (X11; U; Linux 2.2.16-22smp i686; en-US; m18) Gecko/20010110 Netscape6/6.5 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Robert Watson References: <4444EE93.9050003@seclark.us> <44459286.1000008@seclark.us> <20060424141346.O44099@fledge.watson.org> In-Reply-To: <20060424141346.O44099@fledge.watson.org> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Cc: stable@freebsd.org Subject: Re: FreeBSD 4.9 losing mbufs!!! X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Stephen.Clark@seclark.us List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Apr 2006 13:30:48 -0000 Robert Watson wrote: >On Tue, 18 Apr 2006, Stephen Clark wrote: > > > >>I have discovered that if I disable quaqqa/ospfd then I don't lose mbufs! >>This makes it appear that the mbuf leak is in the multicast routing logic. >>In fact I lose mbufs even with the both system basically idle but with a 100 >>vpn/gre with multicast going on thru the gre then the vpn. >> >>Any ideas on where to focus my continued investigation? >> >>Thanks to everybody who has responded. >> >> > >Steve, > >Sorry not to have caught this thread earlier; I've been on travel for the last >few weeks. My general suggestion would be to try to narrow the code paths >traversed to try to eliminate as much code as possible from the search. It >sounds like you've done that pretty effectively :-). > >Typically, memory leaks occur in edge error cases, where the memory is not >properly released, or ownership is unclear. My suggestion would be to add >counters (or look at existing counters where already present) and see if >there's an error case being triggered in about the same quantity that mbuf >leakage is occuring. Chances are, there's an error being returned and a >missing m_freem(). > >Based on your comments above, I might also pay attention to the routing socket >path -- the rate of leak could correspond to the routing daemons talking to >the network stack, rather than the rate of traffic. For example, it could be >that one of the routing messages is handled improperly resulting in a leak. > >Unfortunately, tracking down memory leaks can be quite difficult, and tends to >require a combination of dogged persistence and luck... > >Robert N M Watson >_______________________________________________ >freebsd-stable@freebsd.org mailing list >http://lists.freebsd.org/mailman/listinfo/freebsd-stable >To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > > > Good news and bad news. I managed to get enough of our system running on 6.x stable to test and it does not appear to lose mbufs. Bad news my ipsec transfer rate dropped from 54mbits/sec to 39mbits/sec. We need to be able to handle a t3 (45mbits/sec). Any ideas as to why this drop off in 6.x? Steve -- "They that give up essential liberty to obtain temporary safety, deserve neither liberty nor safety." (Ben Franklin) "The course of history shows that as a government grows, liberty decreases." (Thomas Jefferson)