Skip site navigation (1)Skip section navigation (2)
Date:      Mon, 26 Jan 2009 12:52:46 -0500
From:      gnn@freebsd.org
To:        Rui Paulo <rpaulo@freebsd.org>
Cc:        Liran Liss <liranl@mellanox.co.il>, freebsd-net@freebsd.org, Yony Yossef <yonyossef.lists@gmail.com>, johan@nocrew.org, Amit Krig <amitk@mellanox.co.il>, Eitan Shefi <eitans@mellanox.co.il>
Subject:   Re: freebsd 7.0-RELEASE BUG ping: sendto: No buffer space available
Message-ID:  <7iocxu7y4x.wl%gnn@neville-neil.com>
In-Reply-To: <EDD4152E-8BE7-40C3-A153-21A110152BE8@freebsd.org>
References:  <000001c97e23$0d81df20$39ed1aac@mtl.com> <EDD4152E-8BE7-40C3-A153-21A110152BE8@freebsd.org>

next in thread | previous in thread | raw e-mail | index | archive | help
At Sat, 24 Jan 2009 16:20:06 +0000,
Rui Paulo wrote:
> 
> 
> On 24 Jan 2009, at 12:54, Yony Yossef wrote:
> 
> > Hi All,
> >
> > I'm facing a temporary network hang on my interfaces following a flood
> > ping/stress udp test.
> >
> > I'm running a netperf UDP test which is giving results but does not  
> > return
> > to the shell.
> > client output:
> >
> > UDP UNIDIRECTIONAL SEND TEST from fe80::202:c9ff:fe02:e1fe%mtnic0
> > (fe80::202:c9ff:fe02:e1fe) port 0 AF_INET6 to  
> > fe80::202:c9ff:fe02:e1f4%mt
> > nic0 (fe80::202:c9ff:fe02:e1f4) port 0 AF_INET6
> > Socket  Message  Elapsed      Messages
> > Size    Size     Time         Okay Errors   Throughput
> > bytes   bytes    secs            #      #   10^6bits/sec
> >
> > 32768    1472   10.02      547428 1694280     643.60
> > 32768           10.02       25089             29.50
> >
> >
> > (HANG)
> >
> > After a minute or two it returns to the shell with the following  
> > message:
> > shutdown_control: no response received  errno 55
> >
> > 20 minutes later (!!) the interface is working again.
> >
> > netstat -m and vmstat -z outputs during the hang time:
> >
> > # netstat -m
> > 25687/6578/32265 mbufs in use (current/cache/total)
> > 17404/2438/19842/65536 mbuf clusters in use (current/cache/total/max)
> > 0/1024 mbuf+clusters out of packet secondary zone in use (current/ 
> > cache)
> > 2071/1369/3440/65536 4k (page size) jumbo clusters in use
> > (current/cache/total/max)
> > 0/0/0/65536 9k jumbo clusters in use (current/cache/total/max)
> > 0/0/0/3200 16k jumbo clusters in use (current/cache/total/max)
> > 49513K/11996K/61510K bytes allocated to network (current/cache/total)
> > 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
> > 0/0/0 requests for jumbo clusters denied (4k/9k/16k)
> > 0/0/0 sfbufs in use (current/peak/max)
> > 0 requests for sfbufs denied
> > 0 requests for sfbufs delayed
> > 0 requests for I/O initiated by sendfile
> > 0 calls to protocol drain routines
> 
> I think there are too many mbufs in use. You're probably facing an  
> mbuf leakage and that causes an interface hang.
> 
If this is a large memory machine try upping the number of clusters
and mbufs.  On 64 bit systems with large memories 1,000,000 mbufs is
not unheard of.

kern.ipc.nmbclusters: 1000000

Also, with UDP you can easily overrun different buffers within the
system.  You might also look at:

netstat -id

and see if the driver is dropping packets, and if so you might up its
send queue.  

Best,
George





Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?7iocxu7y4x.wl%gnn>