From owner-freebsd-net@FreeBSD.ORG Mon Aug 26 14:34:16 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 7956CC8F for ; Mon, 26 Aug 2013 14:34:16 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C67662863 for ; Mon, 26 Aug 2013 14:34:15 +0000 (UTC) Received: (qmail 6860 invoked from network); 26 Aug 2013 15:16:26 -0000 Received: from c00l3r.networx.ch (HELO [127.0.0.1]) ([62.48.2.2]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 26 Aug 2013 15:16:26 -0000 Message-ID: <521B6763.1080308@freebsd.org> Date: Mon, 26 Aug 2013 16:34:11 +0200 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130801 Thunderbird/17.0.8 MIME-Version: 1.0 To: Navdeep Parhar Subject: Re: Please review: LRO entry last-active timestamp. References: <521510CC.3040104@FreeBSD.org> In-Reply-To: <521510CC.3040104@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Aug 2013 14:34:16 -0000 On 21.08.2013 21:11, Navdeep Parhar wrote: > I'd like to add a last-active timestamp to the structure that tracks the > LRO state in a NIC's rx handler. This is r254336 in user/np/cxl_tuning > that will be merged to head if there are no objections. No objections. This is good thing. The last time I looked some time back there were a couple of additional issues with the soft-LRO code. One was that it would cause packet re- ordering when a new segment is out-of-order or can't be merged. There the new packet would be delivered first and the merged LRO chain after it. I think there were a couple of other similar issues. However I don't know if these have been fixed in the mean time. BTW an excellent resource on the correct LRO behavior is this detailed flow chart. It probably would be good to audit our soft-LRO against it. http://msdn.microsoft.com/en-us/library/windows/hardware/jj853325%28v=vs.85%29.aspx -- Andre > http://svnweb.freebsd.org/base?view=revision&revision=254336 > http://svnweb.freebsd.org/base/user/np/cxl_tuning/sys/netinet/tcp_lro.c?r1=254336&r2=254335&pathrev=254336 > http://svnweb.freebsd.org/base/user/np/cxl_tuning/sys/netinet/tcp_lro.h?r1=254336&r2=254335&pathrev=254336 > > > ----- > Add a last-modified timestamp to each LRO entry and provide an interface > to flush all inactive entries. Drivers decide when to flush and what > the inactivity threshold should be. > > Network drivers that process an rx queue to completion can enter a > livelock type situation when the rate at which packets are received > reaches equilibrium with the rate at which the rx thread is processing > them. When this happens the final LRO flush (normally when the rx > routine is done) does not occur. Pure ACKs and segments with total > payload < 64K can get stuck in an LRO entry. Symptoms are that TCP > tx-mostly connections' performance falls off a cliff during heavy, > unrelated rx on the interface. > > Flushing only inactive LRO entries works better than any of these > alternates that I tried: > - don't LRO pure ACKs > - flush _all_ LRO entries periodically (every 'x' microseconds or every > 'y' descriptors) > - stop rx processing in the driver periodically and schedule remaining > work for later. > ----- > > > Regards, > Navdeep > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > >