From owner-freebsd-stable@FreeBSD.ORG Fri Nov 25 04:22:11 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C77BA1065674 for ; Fri, 25 Nov 2011 04:22:11 +0000 (UTC) (envelope-from kristoph.bauer@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 7D63A8FC16 for ; Fri, 25 Nov 2011 04:22:11 +0000 (UTC) Received: by iakl21 with SMTP id l21so6028280iak.13 for ; Thu, 24 Nov 2011 20:22:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=b5HDsaxG/GA8L+sGmGzqhJLk7er7n+EvK4Y/YsesoDA=; b=YZ33sDBRHWFDy8WA0ikIpVHLbO7hSkPak0j+oquQCsy98CXBuRROSAF0e7JtggoRLi 4DRNUnRHIgIFBIGIvilulxhaPG2lC+efMahAZzIq20OTDV/dpDVvR9+xYBUZC1WIXNQU WOuquHx41DLfy8nFCmkFG/lsY55cVq9RNnQN4= MIME-Version: 1.0 Received: by 10.43.53.1 with SMTP id vo1mr11217256icb.2.1322194931046; Thu, 24 Nov 2011 20:22:11 -0800 (PST) Received: by 10.50.34.167 with HTTP; Thu, 24 Nov 2011 20:22:11 -0800 (PST) In-Reply-To: <4ECF1796.2060107@freebsd.org> References: <4ECE9914.6020502@turing.b2n.org> <1A5B3A48-7DF3-4018-A244-152BDE96299A@lassitu.de> <20111125020004.GA36109@icarus.home.lan> <4ECF1796.2060107@freebsd.org> Date: Thu, 24 Nov 2011 22:22:11 -0600 Message-ID: From: Kris Bauer To: Lawrence Stewart Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: "freebsd-stable@freebsd.org List" Subject: Re: TCP Reassembly Issues X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Nov 2011 04:22:11 -0000 On Thu, Nov 24, 2011 at 10:20 PM, Lawrence Stewart wrote: > On 11/25/11 14:19, Kris Bauer wrote: > >> On Thu, Nov 24, 2011 at 8:00 PM, Jeremy Chadwick >> wrote: >> >> On Thu, Nov 24, 2011 at 07:13:39PM -0600, Kris Bauer wrote: >>> >>>> On Thu, Nov 24, 2011 at 5:35 PM, Adrian Chadd >>>> >>> wrote: >>> >>>> >>>> Have you tried disabling the tcp offload features of your NIC? >>>>> >>>>> >>>>> Adrian >>>>> >>>>> >>>> To test this, I added net.inet.tcp.tso=0 to sysctl.conf and restarted >>>> the >>>> box; it didn't work. net.inet.tcp.reass.cursegments immediately started >>>> climbing up and were exhausted within an hour. >>>> >>> >>> I think Adrian was referring to RXCSUM and TXCSUM on your NIC; TSO is >>> another offloading feature. >>> >>> See ifconfig(8) for how to disable those. >>> >>> Be aware that disabling them in real-time (e.g. ifconfig xxx -rxcsum >>> -txcsum) may cause problems; there are some NIC drivers on FreeBSD which >>> do not like you doing this once the NIC has established link (meaning >>> "reloading the driver" (for lack of better term) results in wonky >>> behaviour). So you may instead want to add those hyphen-options to your >>> ifconfig_XXX lines in /etc/rc.conf and reboot the box. >>> >>> If none of this solves the problem, then I consider this a priority 0 >>> blocker (read: "all hands on deck") issue with the IP stack in FreeBSD >>> 9.x and will need immediate attention. >>> >>> I would strongly recommend a developer or clueful end-user begin >>> tracking down who committed all of these bits and CC them into the >>> thread. I would start by looking who implemented the >>> net.inet.tcp.reass.cursegments sysctl, because that isn't in RELENG_8 at >>> all. >>> >>> >> I have added -rxcsum -txcsum -tso to rc.conf and rebooted the box. This >> has not solved the problem. After a half-hour usage, I'm already up to >> reass.cursegments=2182 and it keeps climbing. >> > > This is pretty much guaranteed to be an accounting problem in the TCP > reassembly code (netinet/tcp_reass.c), not a driver related issue. I would > not expect any amount of tweaking, tuning or driver option twiddling to > change the outcome (but if you do find something which alleviates it, do > let us know). > > Kris, are you in a position to test kernel patches on the machine which is > experiencing this problem? > > Cheers, > Lawrence > I'd be happy to test kernel patches with this machine. Thanks, Kris