From owner-freebsd-current@freebsd.org Tue Aug 7 23:11:15 2018 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9E529106DC61 for ; Tue, 7 Aug 2018 23:11:15 +0000 (UTC) (envelope-from ambrisko@ambrisko.com) Received: from mail.ambrisko.com (mail.ambrisko.com [70.91.206.90]) by mx1.freebsd.org (Postfix) with ESMTP id DDCED854D7; Tue, 7 Aug 2018 23:11:14 +0000 (UTC) (envelope-from ambrisko@ambrisko.com) X-Ambrisko-Me: Yes Received: from server2.ambrisko.com (HELO internal.ambrisko.com) ([192.168.1.2]) by ironport.ambrisko.com with ESMTP; 07 Aug 2018 16:19:43 -0700 Received: from ambrisko.com (localhost [127.0.0.1]) by internal.ambrisko.com (8.14.7/8.15.2) with ESMTP id w77NBDsG061049 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 7 Aug 2018 16:11:13 -0700 (PDT) (envelope-from ambrisko@ambrisko.com) Received: (from ambrisko@localhost) by ambrisko.com (8.14.7/8.15.2/Submit) id w77NBD8B061048; Tue, 7 Aug 2018 16:11:13 -0700 (PDT) (envelope-from ambrisko) Date: Tue, 7 Aug 2018 16:11:13 -0700 From: Doug Ambrisko To: Konstantin Belousov Cc: Kyle Evans , Eitan Adler , freebsd-current Current Subject: Re: panic: mutex pmap not owned at ... efirt_machdep.c:255 Message-ID: <20180807231113.GA58652@ambrisko.com> References: <20180804083720.GJ6049@kib.kiev.ua> <20180805104341.GX6049@kib.kiev.ua> <20180807172949.GD1884@kib.kiev.ua> <20180807214910.GA53911@ambrisko.com> <20180807224207.GF1884@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20180807224207.GF1884@kib.kiev.ua> User-Agent: Mutt/1.8.3 (2017-05-23) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Aug 2018 23:11:15 -0000 On Wed, Aug 08, 2018 at 01:42:07AM +0300, Konstantin Belousov wrote: | On Tue, Aug 07, 2018 at 02:49:10PM -0700, Doug Ambrisko wrote: | > On Tue, Aug 07, 2018 at 08:29:49PM +0300, Konstantin Belousov wrote: | > | On Tue, Aug 07, 2018 at 11:50:44AM -0500, Kyle Evans wrote: | > | > On Tue, Aug 7, 2018 at 12:09 AM, Eitan Adler wrote: | > | > > On Mon, 6 Aug 2018 at 11:27, Kyle Evans wrote: | > | > >> | > | > >> On Sun, Aug 5, 2018 at 5:43 AM, Konstantin Belousov wrote: | > | > >> > On Sat, Aug 04, 2018 at 09:46:39PM -0500, Kyle Evans wrote: | > | > >> >> | > | > >> >> He now gets a little further, but ends up with the same panic due to | > | > >> >> efirtc_probe trying to get time to verify the rtc's actually | > | > >> >> implemented. What kind of approach must we take to ensure curcpu is | > | > >> >> synced? | > | > >> > | > | > >> > It does not panic for me, when I load efirt.ko from the loader prompt. | > | > >> > Anyway, try this | > | > >> | > | > >> Right, I also don't get a panic on any of my machines from this. | > | > >> Hopefully he'll have a chance to try this soon. | > | > > | > | > > This change has no impact: it still panics in the same way as without the patch. | > | > > | > | > | > | > That seems indicative of a bigger problem, since we use proc0 | > | > throughout all these bits so we should still be dealing with the same | > | > pmap that got passed to pmap_pinit0 when we grab | > | > curthread->td_proc->p_vmspace->vm_pmap. | > | | > | Can you confirm that you get the early efi_enter() call from rtc code, | > | when you preload the module or compile it into the kernel ? | > | > When I ran into this, I did this change: | > | > Index: dev/efidev/efirt.c | > =================================================================== | > --- dev/efidev/efirt.c (revision 337264) | > +++ dev/efidev/efirt.c (working copy) | > @@ -257,7 +257,8 @@ | > if (efi_runtime == NULL) | > return (ENXIO); | > td = curthread; | > - curpmap = &td->td_proc->p_vmspace->vm_pmap; | > +// curpmap = &td->td_proc->p_vmspace->vm_pmap; | > + curpmap = PCPU_GET(curpmap); | > PMAP_LOCK(curpmap); | > mtx_lock(&efi_lock); | > fpu_kern_enter(td, NULL, FPU_KERN_NOCTX); | > @@ -272,7 +273,8 @@ | > | > efi_arch_leave(); | > | > - curpmap = &curproc->p_vmspace->vm_pmap; | > +// curpmap = &curproc->p_vmspace->vm_pmap; | > + curpmap = PCPU_GET(curpmap); | > td = curthread; | > fpu_kern_leave(td, NULL); | > mtx_unlock(&efi_lock); | > | > Don't know if it is right. Some previous code used both | > curpmap = PCPU_GET(curpmap); | > and | > curpmap = &td->td_proc->p_vmspace->vm_pmap; | > recently it was changes to only use | > curpmap = &td->td_proc->p_vmspace->vm_pmap; | > | > Things seem to work after that. I was able to repro. it with | > qemu-system-x86_64 in UEFI mode. I think it also failed in | > bhyve UEFI mode. | | The pcpu curpmap and curproc vmspace pmap should be synced. Esp. since | there is code relying on this early. I do not want to paper it over. | | In fact, try this please. Ignore my previous change. | | diff --git a/sys/amd64/amd64/pmap.c b/sys/amd64/amd64/pmap.c | index 572b2197453..4bce36cc0e5 100644 | --- a/sys/amd64/amd64/pmap.c | +++ b/sys/amd64/amd64/pmap.c | @@ -7536,7 +7536,8 @@ pmap_activate_sw(struct thread *td) | PCPU_SET(kcr3, pmap->pm_cr3); | PCPU_SET(ucr3, pmap->pm_ucr3); | } | - } | + } else | + PCPU_SET(curpmap, pmap); | if (pmap->pm_ucr3 != PMAP_NO_CR3) { | rsp0 = ((vm_offset_t)PCPU_PTR(pti_stack) + | PC_PTI_STACK_SZ * sizeof(uint64_t)) & ~0xful; That works for qemu and bhyve booting in UEFI PXE mode. I backed out my other change and synced to head. Thanks, Doug A.