From owner-freebsd-current Thu Apr 25 11: 7:53 2002 Delivered-To: freebsd-current@freebsd.org Received: from mail.speakeasy.net (mail12.speakeasy.net [216.254.0.212]) by hub.freebsd.org (Postfix) with ESMTP id 74ABF37B405 for ; Thu, 25 Apr 2002 11:07:44 -0700 (PDT) Received: (qmail 1596 invoked from network); 25 Apr 2002 18:07:43 -0000 Received: from unknown (HELO server.baldwin.cx) ([216.27.160.63]) (envelope-sender ) by mail12.speakeasy.net (qmail-ldap-1.03) with DES-CBC3-SHA encrypted SMTP for ; 25 Apr 2002 18:07:43 -0000 Received: from laptop.baldwin.cx (gw1.twc.weather.com [216.133.140.1]) by server.baldwin.cx (8.11.6/8.11.6) with ESMTP id g3PI7mv11291; Thu, 25 Apr 2002 14:07:48 -0400 (EDT) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.5.2 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <20020425110133.A48383@xor.obsecurity.org> Date: Thu, 25 Apr 2002 14:06:51 -0400 (EDT) From: John Baldwin To: Kris Kennaway Subject: Re: 4 panics under 5.0 on ports cluster Cc: current@FreeBSD.org Sender: owner-freebsd-current@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 25-Apr-2002 Kris Kennaway wrote: > On Thu, Apr 25, 2002 at 01:10:50PM -0400, John Baldwin wrote: >> >> On 25-Apr-2002 Kris Kennaway wrote: >> > panic: sleeping thread owns a mutex >> > Uptime: 8h52m59s >> > (kgdb) bt >> >#0 dumpsys () at /local0/scratch/usr/src/sys/kern/kern_shutdown.c:506 >> >#1 0xc0207cb0 in boot (howto=260) at >> >#/local0/scratch/usr/src/sys/kern/kern_shutdown.c:338 >> >#2 0xc020814f in panic (fmt=0xc0367580 "sleeping thread owns a mutex") >> > at /local0/scratch/usr/src/sys/kern/kern_shutdown.c:648 >> >#3 0xc0200a35 in propagate_priority (td=0xcd23d400) >> > at /local0/scratch/usr/src/sys/kern/kern_mutex.c:110 >> >> Can you go to this frame, then dump the contents of 'm' to find the >> misbehaving >> mutex? > > $2 = {mtx_object = {lo_class = 0xc03b1fc0, lo_name = 0xc76785a0 "PCPU 16384", > lo_flags = 4390912, > lo_list = {stqe_next = 0x0}, lo_witness = 0x0}, mtx_lock = 3453968642, > mtx_recurse = 0, > mtx_blocked = {tqh_first = 0xcd23d400, tqh_last = 0xcd23d428}, > mtx_contested = {le_next = 0x0, > le_prev = 0xcddf6160}} Ok, so uma is misbehaving it seems and sleeping while holding a lock. >> WITNESS should have warned about this but I guess getting people to run >> WITNESS is too much to ask. > > WITNESS disables itself after about 30 seconds under load on the ports > cluster..something about some resource becoming exhausted. I tried > increasing the number of whatever resource it was by a factor of 8, > but it was still occurring after a longer time interval, so I gave up > and just turned it off. You have stale kernels. This was fixed a couple of weeks ago with all the mtx_init() changes. If the kernels were up to date, there would be a lo_type in struct lock_object above. Try updating to newer kernels, turning WITNESS and WITNESS_DDB on. Then when uma misbehaves it will drop into ddb. > Kris -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-current" in the body of the message