From owner-freebsd-net@FreeBSD.ORG Mon Aug 11 11:49:27 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D5ABF43A for ; Mon, 11 Aug 2014 11:49:27 +0000 (UTC) Received: from mail-n.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mail-n.franken.de", Issuer "Thawte DV SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6468529CA for ; Mon, 11 Aug 2014 11:49:27 +0000 (UTC) Received: from [192.168.1.200] (p548181A9.dip0.t-ipconnect.de [84.129.129.169]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTP id 59F9D1C0E96E9; Mon, 11 Aug 2014 13:49:23 +0200 (CEST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: A problem on TCP in High RTT Environment. From: Michael Tuexen In-Reply-To: Date: Mon, 11 Aug 2014 13:49:22 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20140809184232.GF83475@funkthat.com> <8AE1AC56-D52F-4F13-AAA3-BB96042B37DD@lurchi.franken.de> <20140809204500.GG83475@funkthat.com> <3F6BC212-4223-4AAC-8668-A27075DC55C2@lurchi.franken.de> <20140810022350.GI83475@funkthat.com> <20140810033212.GL83475@funkthat.com> <17A804F3-BEA6-46F4-887F-B68750618FD9@netapp.com> <0CF85443-26AC-4931-9D00-3396C18C7690@lurchi.franken.de> To: Niu Zhixiong X-Mailer: Apple Mail (2.1878.6) Cc: "freebsd-net@freebsd.org" , John-Mark Gurney , "Eggert, Lars" , Bill Yuan X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Aug 2014 11:49:28 -0000 On 11 Aug 2014, at 09:31, Niu Zhixiong wrote: > During TCP benchmark. >=20 > root@Freetest0: # netstat -m > 35806/2129/37935 mbufs in use (current/cache/total) > 35251/1037/36288/1011638 mbuf clusters in use = (current/cache/total/max) > 35251/1030 mbuf+clusters out of packet secondary zone in use = (current/cache) > 0/6/6/505818 4k (page size) jumbo clusters in use = (current/cache/total/max) > 0/0/0/149872 9k jumbo clusters in use (current/cache/total/max) > 0/0/0/84303 16k jumbo clusters in use (current/cache/total/max) > 79481K/2630K/82111K bytes allocated to network (current/cache/total) > 15/18307/31 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > 0/0/0 requests for jumbo clusters denied (4k/9k/16k) > 0 requests for sfbufs denied > 0 requests for sfbufs delayed > 0 requests for I/O initiated by sendfile >=20 > During SCTP benchmark > 36131/1804/37935 mbufs in use (current/cache/total) > 35394/894/36288/1011638 mbuf clusters in use (current/cache/total/max) > 35394/887 mbuf+clusters out of packet secondary zone in use = (current/cache) > 0/6/6/505818 4k (page size) jumbo clusters in use = (current/cache/total/max) > 0/0/0/149872 9k jumbo clusters in use (current/cache/total/max) > 0/0/0/84303 16k jumbo clusters in use (current/cache/total/max) > 79950K/2263K/82213K bytes allocated to network (current/cache/total) > 15/18307/31 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > 0/0/0 requests for jumbo clusters denied (4k/9k/16k) > 0 requests for sfbufs denied > 0 requests for sfbufs delayed > 0 requests for I/O initiated by sendfile Thanks. I was more asking Lars to have a look if the igb problems he = observed are related to mbuf shortages... I observed that a long time ago and I = think it was fixed. Not sure about 10.0, since this was on head at that = time... Best regards Michael >=20 >=20 > Regards, > Niu Zhixiong > =EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D= =EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D > kaiaixi@gmail.com >=20 >=20 > On Mon, Aug 11, 2014 at 3:17 PM, Michael Tuexen = wrote: > On 11 Aug 2014, at 09:08, Eggert, Lars wrote: >=20 > > Hi, > > > > On 2014-8-10, at 5:48, Niu Zhixiong wrote: > >> I am using Intel I350-T4 NIC. > > > > igb driver? > > > > I've been having weird issues with this driver under 10-RELEASE, = too. On one machine, I had to limit hw.igb.num_queues=3D2 in order to = get any sort of useful connectivity. On another machine, I had to = severely bump kern.ipc.nmbclusters & friends. I'm not sure this is the = issue here, since SCTP seems to be working OK, but I'm not trusting igb = NICs at the moment. > Was there any suspicious output provided by netstat -m when the = problems occur? >=20 > Best regards > Michael > > > > Lars >=20 >=20