From owner-freebsd-current@freebsd.org Tue Aug 7 22:42:27 2018 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DC02F106D0F9 for ; Tue, 7 Aug 2018 22:42:26 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6838A8463D; Tue, 7 Aug 2018 22:42:26 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id w77Mg7RV047569 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 8 Aug 2018 01:42:10 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua w77Mg7RV047569 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id w77Mg7h4047568; Wed, 8 Aug 2018 01:42:07 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 8 Aug 2018 01:42:07 +0300 From: Konstantin Belousov To: Doug Ambrisko Cc: Kyle Evans , Eitan Adler , freebsd-current Current Subject: Re: panic: mutex pmap not owned at ... efirt_machdep.c:255 Message-ID: <20180807224207.GF1884@kib.kiev.ua> References: <20180804083720.GJ6049@kib.kiev.ua> <20180805104341.GX6049@kib.kiev.ua> <20180807172949.GD1884@kib.kiev.ua> <20180807214910.GA53911@ambrisko.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20180807214910.GA53911@ambrisko.com> User-Agent: Mutt/1.10.1 (2018-07-13) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Aug 2018 22:42:27 -0000 On Tue, Aug 07, 2018 at 02:49:10PM -0700, Doug Ambrisko wrote: > On Tue, Aug 07, 2018 at 08:29:49PM +0300, Konstantin Belousov wrote: > | On Tue, Aug 07, 2018 at 11:50:44AM -0500, Kyle Evans wrote: > | > On Tue, Aug 7, 2018 at 12:09 AM, Eitan Adler wrote: > | > > On Mon, 6 Aug 2018 at 11:27, Kyle Evans wrote: > | > >> > | > >> On Sun, Aug 5, 2018 at 5:43 AM, Konstantin Belousov wrote: > | > >> > On Sat, Aug 04, 2018 at 09:46:39PM -0500, Kyle Evans wrote: > | > >> >> > | > >> >> He now gets a little further, but ends up with the same panic due to > | > >> >> efirtc_probe trying to get time to verify the rtc's actually > | > >> >> implemented. What kind of approach must we take to ensure curcpu is > | > >> >> synced? > | > >> > > | > >> > It does not panic for me, when I load efirt.ko from the loader prompt. > | > >> > Anyway, try this > | > >> > | > >> Right, I also don't get a panic on any of my machines from this. > | > >> Hopefully he'll have a chance to try this soon. > | > > > | > > This change has no impact: it still panics in the same way as without the patch. > | > > > | > > | > That seems indicative of a bigger problem, since we use proc0 > | > throughout all these bits so we should still be dealing with the same > | > pmap that got passed to pmap_pinit0 when we grab > | > curthread->td_proc->p_vmspace->vm_pmap. > | > | Can you confirm that you get the early efi_enter() call from rtc code, > | when you preload the module or compile it into the kernel ? > > When I ran into this, I did this change: > > Index: dev/efidev/efirt.c > =================================================================== > --- dev/efidev/efirt.c (revision 337264) > +++ dev/efidev/efirt.c (working copy) > @@ -257,7 +257,8 @@ > if (efi_runtime == NULL) > return (ENXIO); > td = curthread; > - curpmap = &td->td_proc->p_vmspace->vm_pmap; > +// curpmap = &td->td_proc->p_vmspace->vm_pmap; > + curpmap = PCPU_GET(curpmap); > PMAP_LOCK(curpmap); > mtx_lock(&efi_lock); > fpu_kern_enter(td, NULL, FPU_KERN_NOCTX); > @@ -272,7 +273,8 @@ > > efi_arch_leave(); > > - curpmap = &curproc->p_vmspace->vm_pmap; > +// curpmap = &curproc->p_vmspace->vm_pmap; > + curpmap = PCPU_GET(curpmap); > td = curthread; > fpu_kern_leave(td, NULL); > mtx_unlock(&efi_lock); > > Don't know if it is right. Some previous code used both > curpmap = PCPU_GET(curpmap); > and > curpmap = &td->td_proc->p_vmspace->vm_pmap; > recently it was changes to only use > curpmap = &td->td_proc->p_vmspace->vm_pmap; > > Things seem to work after that. I was able to repro. it with > qemu-system-x86_64 in UEFI mode. I think it also failed in > bhyve UEFI mode. The pcpu curpmap and curproc vmspace pmap should be synced. Esp. since there is code relying on this early. I do not want to paper it over. In fact, try this please. Ignore my previous change. diff --git a/sys/amd64/amd64/pmap.c b/sys/amd64/amd64/pmap.c index 572b2197453..4bce36cc0e5 100644 --- a/sys/amd64/amd64/pmap.c +++ b/sys/amd64/amd64/pmap.c @@ -7536,7 +7536,8 @@ pmap_activate_sw(struct thread *td) PCPU_SET(kcr3, pmap->pm_cr3); PCPU_SET(ucr3, pmap->pm_ucr3); } - } + } else + PCPU_SET(curpmap, pmap); if (pmap->pm_ucr3 != PMAP_NO_CR3) { rsp0 = ((vm_offset_t)PCPU_PTR(pti_stack) + PC_PTI_STACK_SZ * sizeof(uint64_t)) & ~0xful;