From owner-freebsd-hackers@FreeBSD.ORG Tue Jun 16 17:58:00 2009 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6A9471065676 for ; Tue, 16 Jun 2009 17:58:00 +0000 (UTC) (envelope-from mel.flynn+fbsd.hackers@mailing.thruhere.net) Received: from mailhub.rachie.is-a-geek.net (rachie.is-a-geek.net [66.230.99.27]) by mx1.freebsd.org (Postfix) with ESMTP id 0CFE28FC0C for ; Tue, 16 Jun 2009 17:57:59 +0000 (UTC) (envelope-from mel.flynn+fbsd.hackers@mailing.thruhere.net) Received: from smoochies.rachie.is-a-geek.net (mailhub.rachie.is-a-geek.net [192.168.2.11]) by mailhub.rachie.is-a-geek.net (Postfix) with ESMTP id EC27E7E842; Tue, 16 Jun 2009 09:57:58 -0800 (AKDT) From: Mel Flynn To: freebsd-hackers@freebsd.org Date: Tue, 16 Jun 2009 09:52:23 -0800 User-Agent: KMail/1.11.4 (FreeBSD/8.0-CURRENT; KDE/4.2.4; i386; ; ) References: <200906151353.06630.mel.flynn+fbsd.hackers@mailing.thruhere.net> <200906160819.57658.jhb@freebsd.org> In-Reply-To: <200906160819.57658.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200906160952.24895.mel.flynn+fbsd.hackers@mailing.thruhere.net> Cc: Subject: Re: How best to debug locking/scheduler problems X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jun 2009 17:58:00 -0000 Hi John, On Tuesday 16 June 2009 04:19:57 John Baldwin wrote: > On Monday 15 June 2009 5:53:05 pm Mel Flynn wrote: > > PID TID COMM TDNAME KSTACK > > 4283 100215 kdeinit4 - mi_switch turnstile_wait > > _mtx_lock_sleep uipc_peeraddr kern_getpeername getpeername syscall > > Xint0x80_syscall > > % ps -ww 4283 > > PID TT STAT TIME COMMAND > > 4283 ?? T 0:00.38 kdeinit4: kdeinit4: kio_http http > > local:/tmp/ksocket-mel/klauncherxJ1635.slave-socket local:/tmp/ksocket- > > mel/plasmayC1653.slave-socket (kdeinit4) > > > > %ls -l /tmp/ksocket-mel/ > > > > total 2 > > -rw-rw-r-- 1 mel wheel 62 Jun 14 22:55 KSMserver__0 > > srw------- 1 mel wheel 0 Jun 14 22:55 kdeinit4__0 > > srwxrwxr-x 1 mel wheel 0 Jun 14 22:55 klauncherxJ1635.slave-socket > > You can use kgdb and the scripts at www.freebsd.org/~jhb/gdb. Simply > run 'kgdb' as root and do 'lcd /folder/with/scripts' and 'source gdb6'. > You can then do 'lockchain 4283' to find who holds the lock this thread is > blocked on and what state they are in. Looks like a deadlock: (kgdb) lockchain 4283 thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx" thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx" thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx" thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx" thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx" thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx" thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx" thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx" thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx" thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx" thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx" thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx" thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx" thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx" thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx" thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx" thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx" thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx" thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx" thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx" DEADLOCK Looking through the scripts now to see how I can get more info on the call chain and hoping I don't panic the machine ;). It is quite random to reproduce. -- Mel