From owner-freebsd-stable@FreeBSD.ORG Thu Aug 11 11:40:33 2011 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B11D21065672; Thu, 11 Aug 2011 11:40:33 +0000 (UTC) (envelope-from prvs=1204ca57bc=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) by mx1.freebsd.org (Postfix) with ESMTP id 106598FC1E; Thu, 11 Aug 2011 11:40:32 +0000 (UTC) X-MDAV-Processed: mail1.multiplay.co.uk, Thu, 11 Aug 2011 12:39:23 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Thu, 11 Aug 2011 12:39:23 +0100 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on mail1.multiplay.co.uk X-Spam-Level: X-Spam-Status: No, score=-5.0 required=6.0 tests=USER_IN_WHITELIST shortcircuit=ham autolearn=disabled version=3.2.5 Received: from r2d2 ([188.220.16.49]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50014559134.msg; Thu, 11 Aug 2011 12:39:21 +0100 X-MDRemoteIP: 188.220.16.49 X-Return-Path: prvs=1204ca57bc=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: From: "Steven Hartland" To: "Andriy Gapon" References: <47F0D04ADF034695BC8B0AC166553371@multiplay.co.uk> <4E4380C0.7070908@FreeBSD.org> Date: Thu, 11 Aug 2011 12:39:52 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6109 Cc: freebsd-stable@FreeBSD.org Subject: Re: debugging frequent kernel panics on 8.2-RELEASE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Aug 2011 11:40:33 -0000 ----- Original Message ----- From: "Andriy Gapon" > on 10/08/2011 18:35 Steven Hartland said the following: >> Fatal double fault ... >> #14 0xffffffff803a2cc9 in sched_switch (td=0x0, newtd=0x0, flags=Variable "flags" >> is not available. >> ) >> at /usr/src/sys/kern/sched_ule.c:1852 >> Previous frame inner to this frame (corrupt stack?) >> (kgdb) > > Looks like this is just the first thread in the kernel. > Perhaps 'thread apply all bt' could help to find the culprit. The trimmed down output, removed the 10,000's of ?? lines here:- http://blog.multiplay.co.uk/dropzone/freebsd/panic-2011-08-11-1402.txt The raw output is here:- http://blog.multiplay.co.uk/dropzone/freebsd/panic-full-2011-08-11-1402.txt.bz2 I'm not sure how useful its going to be as pretty much all of it seems to be just:- #0 sched_tswitch (td=0xffffff00194d4460, newtd=0xffffff000a74a000, flags=Variable "flags" is not available. #1 0xffffffff80385c86 in mi_switch (flags=260, newtd=0x0) at /usr/src/sys/kern/kern_synch.c:449 #2 0xffffffff803b8a0c in sleepq_catch_signals (wchan=0xffffff02f27c48c0, pri=92) at /usr/src/sys/kern/subr_sleepqueue.c:418 #3 0xffffffff803b9326 in sleepq_wait_sig (wchan=Variable "wchan" is not available. #4 0xffffffff80386149 in _sleep (ident=0xffffff02f27c48c0, lock=0xffffff02f27c49b8, priority=Variable "priority" is not available. #5 0xffffffff8035079d in kern_wait (td=0xffffff00194d4460, pid=91362, status=0xffffff86cdbffabc, options=Variable "options" is not available. #6 0xffffffff80350e95 in wait4 (td=Variable "td" is not available. #7 0xffffffff803bb8e5 in syscallenter (td=0xffffff00194d4460, sa=0xffffff86cdbffba0) at /usr/src/sys/kern/subr_trap.c:315 #8 0xffffffff80574a0b in syscall (frame=0xffffff86cdbffc40) at /usr/src/sys/amd64/amd64/trap.c:888 #9 0xffffffff8055d242 in Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:377 On one machine we had a little more info on console which may indicate java as the problem. http://blog.multiplay.co.uk/dropzone/freebsd/panic-java.jpg Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk.