From owner-freebsd-net@FreeBSD.ORG Thu Jan 23 19:49:02 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mandree.no-ip.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9E12A93 for ; Thu, 23 Jan 2014 19:49:01 +0000 (UTC) Received: from [IPv6:::1] (localhost6.localdomain6 [IPv6:::1]) by apollo.emma.line.org (Postfix) with ESMTP id 69FCD23D0FE for ; Thu, 23 Jan 2014 09:20:01 +0100 (CET) Message-ID: <52E0D0B1.3090104@FreeBSD.org> Date: Thu, 23 Jan 2014 09:20:01 +0100 From: Matthias Andree User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: Terrible NFS performance under 9.2-RELEASE? References: <2057911949.13372985.1390269671666.JavaMail.root@uoguelph.ca> In-Reply-To: <2057911949.13372985.1390269671666.JavaMail.root@uoguelph.ca> X-Enigmail-Version: 1.5.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jan 2014 19:49:02 -0000 Am 21.01.2014 03:01, schrieb Rick Macklem: > Since this is getting long winded, I'm going to "cheat" and top post. > (Don't top post flame suit on;-) > > You could try setting > net.inet.tcp.delayed_ack=0 > via sysctl. > I just looked and it appears that TCP delays ACKs for a while, even > when TCP_NODELAY is set (I didn't know that). I honestly don't know > how much/if any effect these delayed ACKs will have, but is you > disable them, you can see what happens. These are separate mechanisms. Not sure if it applies here, but anyways: TCP_NODELAY (opposite is "TCP Cork" or "corking") is for the sending end and means "send data even if you have unacknowledged data in flight and the buffer is not yet full" - and probably does not have much of an impact on bulk sends because you stuff packets full, and then they get sent. Delayed ACK is a feature of the receiving end of the stream so you can bundle return/response data with the ACK (because the "ACK" basically is only any TCP packet with the bumped sequence number in the header). http://www.stuartcheshire.org/papers/NagleDelayedAck/ Mind the warnings at the end of the document - make sure to not impair congestion control and fairness, if you want to avoid harder-to-debug performance issues with mixed traffic in the network (i. e. more than your synthetic NFS tests or NFS-dominated use). It'd be interesting to see if there has been research on the Minshall modification vs. TCP fairness (TCP fairness is paramount through all the TCP research papers, but I have not looked at that for a few years now, and I predominantly looked at other aspects of congestion control at the time). Links (not necessary URLs) welcome, especially if not delaying acks turns out to improve the situation.