From owner-freebsd-virtualization@FreeBSD.ORG  Sun Jul  6 16:28:08 2014
Return-Path: <owner-freebsd-virtualization@FreeBSD.ORG>
Delivered-To: virtualization@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTPS id EA031C02;
 Sun,  6 Jul 2014 16:28:08 +0000 (UTC)
Received: from mail-ob0-x230.google.com (mail-ob0-x230.google.com
 [IPv6:2607:f8b0:4003:c01::230])
 (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits))
 (Client CN "smtp.gmail.com",
 Issuer "Google Internet Authority G2" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 9E5362F43;
 Sun,  6 Jul 2014 16:28:08 +0000 (UTC)
Received: by mail-ob0-f176.google.com with SMTP id wm4so3507674obc.7
 for <multiple recipients>; Sun, 06 Jul 2014 09:28:08 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
 h=mime-version:in-reply-to:references:date:message-id:subject:from:to
 :cc:content-type;
 bh=K+p0NqvRof+s7RlnKnG60FYptpbpmCPqYXAd+kWYKpI=;
 b=UOUbY1ZJx1hTVCRll2RwLbl0u/n5Ll/uiOpD6553kXO6LwwjNPgWmgqRaFJVs1KZAK
 LkDGMDkzfkmMfaREBYwCqhdvwjXLxY/RBqMZ1ss2sI4pY5StelS+8Ehl/iJedjpWoPKY
 wZKiKxUfPMQLYQWAOamItTYenSYLT1azIDG/AixiPWRxDXEy62wGR/yNmBDhO/bwkIAc
 poT0P7rEaHXUZxACMSt5WxjqCDpBpRV6mnsbABkn48+MQoVQWarJVkgIfFMI1O4iryMj
 inVPyEQfOehgZ02isNvulc4dOD90Qa49G/92+HPaC8fKbkyzqb2hxdDQk2BsXuumw6Rh
 fK5A==
MIME-Version: 1.0
X-Received: by 10.60.155.231 with SMTP id vz7mr26097732oeb.56.1404664087919;
 Sun, 06 Jul 2014 09:28:07 -0700 (PDT)
Received: by 10.76.151.227 with HTTP; Sun, 6 Jul 2014 09:28:07 -0700 (PDT)
In-Reply-To: <20140706154621.GA81830@mouf.net>
References: <20140706135333.GA80856@mouf.net> <20140706154621.GA81830@mouf.net>
Date: Sun, 6 Jul 2014 12:28:07 -0400
Message-ID: <CAFMmRNzTFOVBSoU+CMnnEJ_rUooLC4v742hetMtXWMu_RmPzYw@mail.gmail.com>
Subject: Re: tmpfs panic
From: Ryan Stone <rysto32@gmail.com>
To: Steve Wills <swills@freebsd.org>
Content-Type: text/plain; charset=UTF-8
Cc: virtualization@freebsd.org, FreeBSD Current <current@freebsd.org>
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
 <freebsd-virtualization.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-virtualization>, 
 <mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>, 
 <mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sun, 06 Jul 2014 16:28:09 -0000

On Sun, Jul 6, 2014 at 11:46 AM, Steve Wills <swills@freebsd.org> wrote:
> I should have noted this system is running in bhyve. Also I'm told this panic
> may be related to the fact that the system is running in bhyve.
>
> Looking at it a little more closely:
>
> (kgdb) list *__mtx_lock_sleep+0xb1
> 0xffffffff809638d1 is in __mtx_lock_sleep (/usr/src/sys/kern/kern_mutex.c:431).
> 426                      * owner stops running or the state of the lock changes.
> 427                      */
> 428                     v = m->mtx_lock;
> 429                     if (v != MTX_UNOWNED) {
> 430                             owner = (struct thread *)(v & ~MTX_FLAGMASK);
> 431                             if (TD_IS_RUNNING(owner)) {
> 432                                     if (LOCK_LOG_TEST(&m->lock_object, 0))
> 433                                             CTR3(KTR_LOCK,
> 434                                                 "%s: spinning on %p held by %p",
> 435                                                 __func__, m, owner);
> (kgdb)
>
> I'm told that MTX_CONTESTED was set on the unlocked mtx and that MTX_CONTENDED
> is spuriously left behind, and to ask how lock prefix is handled in bhyve. Any
> of that make sense to anyone?

The mutex has both MTX_CONTESTED and MTX_UNOWNED set on it?  That is a
special sentinel value that is set on a mutex when it is destroyed
(see MTX_DESTROYED in sys/mutex.h).  If that is the case it looks like
you've stumbled upon some kind of use-after-free in tmpfs.  I doubt
that bhyve is responsible (other than perhaps changing the timing
around making the panic more likely to happen).