Date: Wed, 25 Jun 2008 21:17:26 +0300 From: "Vlad GALU" <dudu@dudu.ro> To: "Ali Niknam" <freebsd-net@transip.nl> Cc: net@freebsd.org Subject: Re: FreeBSD 7.0: sockets stuck in CLOSED state... Message-ID: <ad79ad6b0806251117s7f183caawdb027e4b7994436b@mail.gmail.com> In-Reply-To: <486283B0.3060805@transip.nl> References: <486283B0.3060805@transip.nl>
next in thread | previous in thread | raw e-mail | index | archive | help
On 6/25/08, Ali Niknam <freebsd-net@transip.nl> wrote: > Dear All, > > Recently i've been upgrading some of my machines from FreeBSD 6.x amd64 to > FreeBSD 7.0 amd64. > > After upgrading I noticed a weird error/bug. It seems that after several > thousand TCP connections some seem to hang in 'CLOSED' state. > > netstat -n gives: > ... > tcp4 0 0 1.2.3.4.* 4.5.6.7.42149 CLOSED > tcp4 39 0 1.2.3.4.* 4.5.6.7.54103 CLOSED > tcp4 35 0 1.2.3.4.* 4.5.6.7.41718 CLOSED > tcp4 38 0 1.2.3.4.* 4.5.6.7.55618 CLOSED > tcp4 41 0 1.2.3.4.* 4.5.6.7.44230 CLOSED > tcp4 39 0 1.2.3.4.* 4.5.6.7.49439 CLOSED > ... > > These never go away; they gradually increase and increase until the > application starts giving errors (probably because some socket or > filedescriptor limit is reached). When the application is killed these > entries disappear. > > The application in question is a self written DNS server, multithreaded, > and running fine for years without any troubles on both BSD 5.x as well as > 6.x. Also 32bits as well as 64bits on 6.x. > > Ofcourse that doesn't mean that the application is error free, however, > after doing extensive testing I really can not find anything wrong with the > application itself, so I'm thinking maybe there's a change somewhere that > causes this? I know that tcp/network has been completely redone... > > What basically happens in the application is this: > - one main tcp thread runs an infinite while loop waiting for new > connections to arrive > - as soon as one arrives a new thread is spawned that handles the newly > created stream > - it reads some bytes, writes some bytes, then closes it > - thread exits > > What appears to happen is this: after the new thread is spawned it tries to > read 2 bytes (DNS tcp length information). It gets back 0 bytes (EOF) and > therefore closes the sockets and calls pthread_exit. However in netstat that > same stream oftenly appears to have bytes 'stuck' in the in queue... > > I really can't see how this can cause hanging sockets in 'CLOSED' state. > Even if the incoming queue isnt read entirely a call to close should close > it. Also I really can't find any documentation in netstat, or elsewhere, > about the 'CLOSED' state... > > > Any help would greatly be appreciated! > > > Kind Regards, > > > Ali Niknam > _______________________________________________ This looks like an issue we used to have at work, where a streaming application suddenly started getting kevents for sockets that had been already closed. While that was happening, a netstat output looked just like yours. We never tracked it down, as we moved to other projects :( -- ~/.signature: no such file or directory
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?ad79ad6b0806251117s7f183caawdb027e4b7994436b>