Date: Thu, 04 Dec 2014 22:00:08 +0300 From: "Alexander V. Chernikov" <melifaro@FreeBSD.org> To: Yuriy Tabolin <tabolin@speechpro.com>, freebsd-net@freebsd.org Subject: Re: problem with 9k jumbo clusters Message-ID: <5480AF38.3070100@FreeBSD.org> In-Reply-To: <54803C75.2020300@speechpro.com> References: <54803C75.2020300@speechpro.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 04.12.2014 13:50, Yuriy Tabolin wrote: > Hi All. > I have a server with two Intel 10G NIC. OS FreeBSD 10.1-Release amd64. > Server works like NFS, samba-server and iSCSI target. Both NICs > aggregated into lagg device and set MTU 9014 to them. There are some > tuning sysctl.conf: > kern.maxfiles=6289601 > kern.maxfilesperproc=5660640 > kern.maxvnodes=3339565 > kern.ipc.nmbclusters=12255588 > kern.ipc.nmbjumbop=6127794 > kern.ipc.nmbufs=78435780 > kern.ipc.maxsockbuf=16777216 > kern.ipc.maxsockets=6289600 > net.inet.tcp.sendbuf_max=16777216 > net.inet.tcp.sendspace=65536 > net.inet.tcp.recvbuf_max=16777216 > net.inet.tcp.recvspace=65536 > > After some days of working, the errors are appearing: > ix1: Interface stopped DISTRIBUTING, possible flapping > ix0: Interface stopped DISTRIBUTING, possible flapping > ix0: Could not setup receive structures > ix1: Could not setup receive structures Hello. It looks like https://lists.freebsd.org/pipermail/freebsd-net/2014-May/038630.html is relevant here. > > After that errors the NICs stoped working. netstat -m shows: > 32881/33854/66735 mbufs in use (current/cache/total) > 16370/8198/24568/12255588 mbuf clusters in use (current/cache/total/max) > 16370/4807 mbuf+clusters out of packet secondary zone in use > (current/cache) > 0/873/873/6127794 4k (page size) jumbo clusters in use > (current/cache/total/max) > 16383/21517/37900/1815641 9k jumbo clusters in use > (current/cache/total/max) > 0/0/0/1021298 16k jumbo clusters in use (current/cache/total/max) > 188407K/222004K/410411K bytes allocated to network (current/cache/total) > 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > 0/101414306/0 requests for jumbo clusters denied (4k/9k/16k) > 0 requests for sfbufs denied > 0 requests for sfbufs delayed > 0 requests for I/O initiated by sendfile > > 9k jumbo clusters max is too big, but looks like system cannot > allocate them. There are huge number of "9k requests for jumbo > clusters denied". ifconfig ix down/up don't helped, reboot is needed. > Thanks for any help! > > > -- > Best regards, > Tabolin Yuriy > System administrator > Speech Technology Center > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5480AF38.3070100>