From owner-freebsd-net@FreeBSD.ORG  Thu Jul 11 13:35:07 2013
Return-Path: <owner-freebsd-net@FreeBSD.ORG>
Delivered-To: freebsd-net@FreeBSD.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 by hub.freebsd.org (Postfix) with ESMTP id 047AAA2A;
 Thu, 11 Jul 2013 13:35:07 +0000 (UTC)
 (envelope-from glebius@FreeBSD.org)
Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10])
 by mx1.freebsd.org (Postfix) with ESMTP id 836051E19;
 Thu, 11 Jul 2013 13:35:05 +0000 (UTC)
Received: from cell.glebius.int.ru (localhost [127.0.0.1])
 by cell.glebius.int.ru (8.14.7/8.14.7) with ESMTP id r6BDZ4VH008721;
 Thu, 11 Jul 2013 17:35:04 +0400 (MSK)
 (envelope-from glebius@FreeBSD.org)
Received: (from glebius@localhost)
 by cell.glebius.int.ru (8.14.7/8.14.7/Submit) id r6BDZ4Ta008720;
 Thu, 11 Jul 2013 17:35:04 +0400 (MSK)
 (envelope-from glebius@FreeBSD.org)
X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to
 glebius@FreeBSD.org using -f
Date: Thu, 11 Jul 2013 17:35:04 +0400
From: Gleb Smirnoff <glebius@FreeBSD.org>
To: Andre Oppermann <andre@freebsd.org>
Subject: Re: Listen queue overflow: N already in queue awaiting acceptance
Message-ID: <20130711133504.GB67810@FreeBSD.org>
References: <51DE591E.7040405@FreeBSD.org>
 <51DE5C8C.3090404@freebsd.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=koi8-r
Content-Disposition: inline
In-Reply-To: <51DE5C8C.3090404@freebsd.org>
User-Agent: Mutt/1.5.21 (2010-09-15)
Cc: freebsd-net@FreeBSD.org, Andriy Gapon <avg@FreeBSD.org>
X-BeenThere: freebsd-net@freebsd.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net>
List-Post: <mailto:freebsd-net@freebsd.org>
List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 11 Jul 2013 13:35:07 -0000

On Thu, Jul 11, 2013 at 09:19:40AM +0200, Andre Oppermann wrote:
A> On 11.07.2013 09:05, Andriy Gapon wrote:
A> > kernel: sonewconn: pcb 0xfffffe0047db3930: Listen queue overflow: 193 already in
A> > queue awaiting acceptance
A> > last message repeated 113 times
A> > last message repeated 518 times
A> > last message repeated 2413 times
A> > last message repeated 2041 times
A> > last message repeated 1741 times
A> > last message repeated 1543 times
A> > last message repeated 1283 times
A> > last message repeated 1178 times
A> > last message repeated 1020 times
A> > ...
A> >
A> > What does this messages mean?
A> 
A> That your server process lagging behind in accepting new connections and a
A> quite a number of them get aborted due to a backlogged listen queue.
A> 
A> Making the accept queue longer doesn't help, it's user-space that can't keep
A> up with the rate of new incoming connections.
A> 
A> You can either reduce the rate of new incoming connections, optimize your
A> server process to accept more connections in the same time, or get a beefier
A> machine.
A> 
A> > Is it really that important to be printed?
A> 
A> The log messages are at DEBUG level.  People probably want to know about
A> their server not keeping up and throwing incoming connection attempts away.
A> 
A> > Finally, why is it not throttled?
A> 
A> The frequency it happens with is important to determine if this is only
A> a temporary spike (micro-burst) or persistent condition.

IMO, this should be a single counter accessible via sysctl, with no
printf(). Those, who need details on whether this is micro-burst or
persistent condition, can run monitoring software that draws plots.

-- 
Totus tuus, Glebius.