From owner-freebsd-net@FreeBSD.ORG Wed Apr 4 17:09:23 2007 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 426AA16A406 for ; Wed, 4 Apr 2007 17:09:23 +0000 (UTC) (envelope-from cswiger@mac.com) Received: from pi.codefab.com (pi.codefab.com [199.103.21.227]) by mx1.freebsd.org (Postfix) with ESMTP id 15F3C13C45E for ; Wed, 4 Apr 2007 17:09:23 +0000 (UTC) (envelope-from cswiger@mac.com) Received: from localhost (localhost [127.0.0.1]) by pi.codefab.com (Postfix) with ESMTP id 727075D56; Wed, 4 Apr 2007 12:43:27 -0400 (EDT) X-Virus-Scanned: amavisd-new at codefab.com Received: from pi.codefab.com ([127.0.0.1]) by localhost (pi.codefab.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0KbAJi5U61MQ; Wed, 4 Apr 2007 12:43:22 -0400 (EDT) Received: from [192.168.1.3] (pool-68-161-116-136.ny325.east.verizon.net [68.161.116.136]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by pi.codefab.com (Postfix) with ESMTP id 5E5235C54; Wed, 4 Apr 2007 12:43:22 -0400 (EDT) Message-ID: <4613D5A5.3000900@mac.com> Date: Wed, 04 Apr 2007 12:43:17 -0400 From: Chuck Swiger User-Agent: Thunderbird 1.5.0.10 (Windows/20070221) MIME-Version: 1.0 To: stefan.lambrev@sun-fish.com References: <461365C2.9080609@sun-fish.com> In-Reply-To: <461365C2.9080609@sun-fish.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org Subject: Re: sockets without owner. X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Apr 2007 17:09:23 -0000 Stefan Lambrev wrote: > I'm having very strange problem. > I have near 200 sockets reported by netstat -An, which are NOT reported > by sockstat and fstat. > All of them look like (output from netstat -An) : > > ffffff0169282000 tcp4 0 0 192.168.13.12.4965 > 192.168.13.3.8080 FIN_WAIT_2 > > I'm trying to figure out what keeps them active (tcpdump shows traffic > from 192.168.13.12 to 192.168.13.3) > The application that create them is stopped before more then a week but > request are still "flying" around. FIN_WAIT_2 means that one side has closed the TCP connection and is waiting for the other side to finish sending any remaining data and acknowledge the close of the connection by sending a FIN. The TCP stack can remain in this state until the other side acknowledges or resets the connection, even though the local process is gone. (In fact, it's not unusual for a process to close and then quit.) The TCP stack is supposed to wait 2*MSL (ie, 120 seconds), and then move to TIME_WAIT and then CLOSED, IIRC, if it doesn't hear anything back. > I have access to both IPs - 192.168.13.12 is http balancer and > 192.168.13.3 is apache server (both were restarted) Sounds like a glich in the TCP state engine of the load-balancer, probably. This sort of thing happens when using LBs somewhat frequently.... -- -Chuck