From owner-freebsd-hackers@FreeBSD.ORG  Tue Apr  8 09:04:27 2014
Return-Path: <owner-freebsd-hackers@FreeBSD.ORG>
Delivered-To: freebsd-hackers@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115])
 (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTPS id 3AACDDC1;
 Tue,  8 Apr 2014 09:04:27 +0000 (UTC)
Received: from mail.tdx.com (mail.tdx.com [62.13.128.18])
 by mx1.freebsd.org (Postfix) with ESMTP id D436017B1;
 Tue,  8 Apr 2014 09:04:26 +0000 (UTC)
Received: from Mail-PC.tdx.co.uk (storm.tdx.co.uk [62.13.130.251])
 (authenticated bits=0)
 by mail.tdx.com (8.14.3/8.14.3/) with ESMTP id s3894O0p023381
 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO);
 Tue, 8 Apr 2014 10:04:25 +0100 (BST)
Date: Tue, 08 Apr 2014 10:04:24 +0100
From: Karl Pielorz <kpielorz_lst@tdx.co.uk>
To: John Baldwin <jhb@freebsd.org>
Subject: Re: Stuck CLOSED sockets / sshd / zombies...
Message-ID: <9647C5438B5CD4A3058AB1A2@Mail-PC.tdx.co.uk>
In-Reply-To: <201404071148.10157.jhb@freebsd.org>
References: <3FE645E9723756F22EF901AE@Mail-PC.tdx.co.uk>
 <201404041613.09808.jhb@freebsd.org>
 <2C5B099DE2229F0E8D82D8C8@Mail-PC.tdx.co.uk>
 <201404071148.10157.jhb@freebsd.org>
X-Mailer: Mulberry/4.0.8 (Win32)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
Cc: freebsd-hackers@freebsd.org
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
 <freebsd-hackers.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-hackers>, 
 <mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers/>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
 <mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 08 Apr 2014 09:04:27 -0000



--On 07 April 2014 11:48 -0400 John Baldwin <jhb@freebsd.org> wrote:

> Ok, do you have a matching /usr/src on the boxes in question?  If so,
> please  do this:
>
> cd /usr/src/lib/libc
> make DEBUG_FLAGS=-g all install

That just installs stuff, not rebuild it right? (Looks to have installed 
non-stripped versions?).

> Then re-run the scan to get a stuck sshd.  Once that happens, please
> attach to the top-most stock sshd (the one in "urdlck") with gdb
> (gdb /usr/sbin/sshd <pid>) and run 'bt' and reply with the output.

Ok, that gives:

"
(gdb) bt
#0  0x00000008038ea89c in _umtx_op_err () from /lib/libthr.so.3
#1  0x00000008038e104f in __thr_rwlock_rdlock () from /lib/libthr.so.3
#2  0x00000008038e821c in _thr_rtld_init () from /lib/libthr.so.3
#3  0x000000080064f9a2 in _rtld_get_stack_prot () from /libexec/ld-elf.so.1
#4  0x00000008006498c9 in r_debug_state () from /libexec/ld-elf.so.1
#5  0x00000008006470cd in .text () from /libexec/ld-elf.so.1
#6  0x0000000000000246 in ?? ()
#7  0x0000000000000000 in ?? ()
"

-Karl