Date: Thu, 5 Apr 2001 19:37:43 -0500 (CDT) From: Chris Dillon <cdillon@wolves.k12.mo.us> To: Archie Cobbs <archie@dellroad.org> Cc: Archie Cobbs <archie@packetdesign.com>, <freebsd-net@FreeBSD.ORG>, <freebsd-stable@FreeBSD.ORG> Subject: Re: mbuf leak? fxp? Message-ID: <Pine.BSF.4.32.0104051844240.54885-100000@mail.wolves.k12.mo.us> In-Reply-To: <200104051718.f35HIcF73652@arch20m.dellroad.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, 5 Apr 2001, Archie Cobbs wrote:
> Archie Cobbs writes:
> > I have this machine that starts running out of mbufs every few days
> > ("looutput: mbuf allocation failed") and then crashes, and was wondering
> > if anyone else has seen similar behavior...
> >
> > For example...
> >
> > Yesterday...
> > $ netstat -m
> > 461/624/4096 mbufs in use (current/peak/max):
> > 459 mbufs allocated to data
> > 2 mbufs allocated to packet headers
> > 434/490/1024 mbuf clusters in use (current/peak/max)
> > 1136 Kbytes allocated to network (36% of mb_map in use)
> > 0 requests for memory denied
> > 0 requests for memory delayed
> > 0 calls to protocol drain routines
> >
> > Today...
> > $ netstat -m
> > 947/1072/4096 mbufs in use (current/peak/max):
> > 945 mbufs allocated to data
> > 2 mbufs allocated to packet headers
> > 920/946/1024 mbuf clusters in use (current/peak/max)
> > 2160 Kbytes allocated to network (70% of mb_map in use)
> > 0 requests for memory denied
> > 0 requests for memory delayed
> > 0 calls to protocol drain routines
> >
> > It appears that something is slowly eating up mbuf clusters.
> > The machine is on a network with continuous but very low volume
> > traffic, including some random multicast, NTP, etc. The machine
> > itself is doing hardly anything at all.
>
> Well, my current guess is that this is simply an NMBCLUSTERS problem.
> I increased NMBCLUSTERS to 8192 and it hasn't happened again yet.
>
> This machine has 5 ethernet interfaces, which must be probably more
> than the default NMBCLUSTERS can handle.
Just a datapoint... I'm running a 4.3-BETA box with 8 fxp interfaces
all on 100Mbit networks (several heavily trafficed, others spurious)
and MAXUSERS set to 128, which gives me 2560 mbuf clusters:
565/2784/10240 mbufs in use (current/peak/max):
537 mbufs allocated to data
28 mbufs allocated to packet headers
524/2038/2560 mbuf clusters in use (current/peak/max)
4772 Kbytes allocated to network (62% of mb_map in use)
0 requests for memory denied
0 requests for memory delayed
0 calls to protocol drain routines
Could probably use a few more mbuf clusters, since its getting
close, but read on...
<proud><grin="wide">
This box has been up for 22 days (been up for many moons before, but I
wanted to test 4.3-BETA on it... yeah, its an "old" BETA already), and
does LOTS of stuff in addition to routing across the 8 fxp interfaces,
including ipfw with over 60 static rules and many hundreds of dynamic
rules, just a little bit of NAT using natd, arpwatch and snort on
about five of the interfaces, and Squid as a HTTP proxy with about
30GB of cache doing about 30000 requests/hour on average (handles
about 60000 requests during the peak hour -- lunchtime). It still has
plenty of power left over to run a distrubuted.net personal proxy and
chew on lots of RC5 keys as well (I love FreeBSD). :-)
</proud></grin>
Its doing pretty much the gamut of network related abuse you could do
to a box -- routing on lots of interfaces, bpfilter (two per interface
in most cases), ipfw, NAT, a fair amount of incoming and outgoing
connections -- except I'm not doing anything Netgraph related
(assuming you might be, being one who wrote it). Maybe its related to
that?
-- Chris Dillon - cdillon@wolves.k12.mo.us - cdillon@inter-linc.net
FreeBSD: The fastest and most stable server OS on the planet.
For IA32 and Alpha architectures. IA64, PPC, and ARM under development.
http://www.freebsd.org
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-stable" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.BSF.4.32.0104051844240.54885-100000>