Date: Sat, 6 Jul 2019 19:49:15 -0700 From: Mark Millard <marklmi@yahoo.com> To: FreeBSD Current <freebsd-current@freebsd.org>, "freebsd-amd64@freebsd.org" <freebsd-amd64@freebsd.org> Subject: amd64 head -r349794 (under Hyper-V): "panic: spin lock held too long" during a buildworld buildkernel Message-ID: <5AA725CF-2470-4C79-A329-951675390BF3@yahoo.com>
next in thread | raw e-mail | index | archive | help
Looks like pmap_invalidate_range using smp_targeted_tlb_shootdown using = _mtx_lock_spin_cookie. I'll note that I had no trouble with -r349444 building world or kernel = repeatedly, including when I originally build -r349794 to upgrade. The below is from my 2nd buildworld buildkernel under -r349794 (but the = 2 builds were not from scratch ones). # more /var/crash/core.txt.2=20 FBSDFSSD dumped core - see /var/crash/vmcore.2 Sun Jul 7 02:26:36 PDT 2019 FreeBSD FBSDFSSD 13.0-CURRENT FreeBSD 13.0-CURRENT #24 r349794M: Sun Jul = 7 01:55:57 PDT 2019 = markmi@FBSDFSSD:/usr/obj/amd64_clang/amd64.amd64/usr/src/amd64.amd64/sys/G= ENERIC-NODBG amd64 panic: spin lock held too long GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you = are welcome to change it and/or distribute copies of it under certain = conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for = details. This GDB was configured as "amd64-marcel-freebsd"... Unread portion of the kernel message buffer: spin lock 0xffffffff829540e0 (smp rendezvous) held by 0xfffff80ae7ebb5a0 = (tid 100669) too long timeout stopping cpus panic: spin lock held too long cpuid =3D 15 time =3D 1562491248 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame = 0xfffffe00db5263c0 vpanic() at vpanic+0x19d/frame 0xfffffe00db526410 panic() at panic+0x43/frame 0xfffffe00db526470 _mtx_lock_indefinite_check() at _mtx_lock_indefinite_check+0x6d/frame = 0xfffffe00db526480 _mtx_lock_spin_cookie() at _mtx_lock_spin_cookie+0xd5/frame = 0xfffffe00db5264f0 smp_targeted_tlb_shootdown() at smp_targeted_tlb_shootdown+0x3de/frame = 0xfffffe00db526560 pmap_invalidate_range() at pmap_invalidate_range+0x25c/frame = 0xfffffe00db5265f0 vm_thread_stack_dispose() at vm_thread_stack_dispose+0x2c/frame = 0xfffffe00db526640 thread_reap() at thread_reap+0x106/frame 0xfffffe00db526660 proc_reap() at proc_reap+0x788/frame 0xfffffe00db5266a0 proc_to_reap() at proc_to_reap+0x463/frame 0xfffffe00db5266f0 kern_wait6() at kern_wait6+0x34c/frame 0xfffffe00db526790 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00db526980 amd64_syscall() at amd64_syscall+0x36e/frame 0xfffffe00db526ab0 fast_syscall_common() at fast_syscall_common+0x101/frame = 0xfffffe00db526ab0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip =3D 0x80038f7fa, rsp =3D = 0x7fffffffb168, rbp =3D 0x7fffffffb1b0 --- KDB: enter: panic Reading symbols from /boot/kernel/intpm.ko...Reading symbols from = /usr/lib/debug//boot/kernel/intpm.ko.debug...done. done. Loaded symbols for /boot/kernel/intpm.ko Reading symbols from /boot/kernel/smbus.ko...Reading symbols from = /usr/lib/debug//boot/kernel/smbus.ko.debug...done. done. Loaded symbols for /boot/kernel/smbus.ko Reading symbols from /boot/kernel/mac_ntpd.ko...Reading symbols from = /usr/lib/debug//boot/kernel/mac_ntpd.ko.debug...done. done. Loaded symbols for /boot/kernel/mac_ntpd.ko Reading symbols from /boot/kernel/imgact_binmisc.ko...Reading symbols = from /usr/lib/debug//boot/kernel/imgact_binmisc.ko.debug...done. done. Loaded symbols for /boot/kernel/imgact_binmisc.ko Reading symbols from /boot/kernel/filemon.ko...Reading symbols from = /usr/lib/debug//boot/kernel/filemon.ko.debug...done. done. Loaded symbols for /boot/kernel/filemon.ko #0 doadump (textdump=3D0) at src/sys/amd64/include/pcpu.h:246 246 __asm("movq %%gs:%P1,%0" : "=3Dr" (td) : "n" = (OFFSETOF_CURTHREAD)); (kgdb) #0 doadump (textdump=3D0) at src/sys/amd64/include/pcpu.h:246 #1 0xffffffff804a152b in db_dump (dummy=3D<value optimized out>,=20 dummy3=3D<value optimized out>, dummy4=3D<value optimized out>) at /usr/src/sys/ddb/db_command.c:575 #2 0xffffffff804a12f9 in db_command (cmd_table=3D<value optimized out>,=20= dopager=3D1) at /usr/src/sys/ddb/db_command.c:482 #3 0xffffffff804a1074 in db_command_loop () at /usr/src/sys/ddb/db_command.c:535 #4 0xffffffff804a42cf in db_trap (type=3D<value optimized out>,=20 code=3D<value optimized out>) at /usr/src/sys/ddb/db_main.c:252 #5 0xffffffff80c502bc in kdb_trap (type=3D3, code=3D0, tf=3D<value = optimized out>) at /usr/src/sys/kern/subr_kdb.c:692 #6 0xffffffff810e1e2c in trap (frame=3D0xfffffe00db5262f0) at /usr/src/sys/amd64/amd64/trap.c:621 #7 0xffffffff810bb8b5 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:232 #8 0xffffffff80c4f9cb in kdb_enter (why=3D0xffffffff8134eb81 "panic",=20= msg=3D<value optimized out>) at src/sys/amd64/include/cpufunc.h:65 #9 0xffffffff80c0226a in vpanic (fmt=3D<value optimized out>,=20 ap=3D<value optimized out>) at /usr/src/sys/kern/kern_shutdown.c:894 #10 0xffffffff80c020a3 in panic (fmt=3D<value optimized out>) at /usr/src/sys/kern/kern_shutdown.c:832 #11 0xffffffff80be128d in _mtx_lock_indefinite_check (m=3D<value = optimized out>,=20 ldap=3D<value optimized out>) at /usr/src/sys/kern/kern_mutex.c:1222 #12 0xffffffff80be0dd5 in _mtx_lock_spin_cookie (c=3D0xffffffff829540f8,=20= v=3D<value optimized out>) at /usr/src/sys/kern/kern_mutex.c:748 #13 0xffffffff8125b32e in smp_targeted_tlb_shootdown (mask=3D {__bits =3D 0xfffffe00db526570}, vector=3D246, = pmap=3D0xffffffff82a682f8,=20 addr1=3D18446741878369513472, addr2=3D18446741878369529856) at /usr/src/sys/x86/x86/mp_x86.c:1671 #14 0xffffffff810cae6c in pmap_invalidate_range = (pmap=3D0xffffffff82a682f8,=20 sva=3D18446741878369513472, eva=3D18446741878369529856) at /usr/src/sys/amd64/amd64/pmap.c:2425 #15 0xffffffff80f474fc in vm_thread_stack_dispose = (ksobj=3D0xfffff80123acdb00,=20 ks=3D18446741878369513472, pages=3D4) at = /usr/src/sys/vm/vm_glue.c:404 #16 0xffffffff80c16796 in thread_reap () at = /usr/src/sys/kern/kern_thread.c:449 #17 0xffffffff80bbb628 in proc_reap (td=3D<value optimized out>,=20 p=3D0xfffff8010b99ea60, status=3D<value optimized out>, options=3D48) at /usr/src/sys/kern/kern_exit.c:969 #18 0xffffffff80bbbd03 in proc_to_reap (td=3D0xfffff80a0d384000,=20 p=3D0xfffff8010b99ea60, idtype=3D<value optimized out>,=20 id=3D<value optimized out>, status=3D<value optimized out>,=20 options=3D<value optimized out>, wrusage=3D0x0, siginfo=3D0x0, = check_only=3D0) at /usr/src/sys/kern/kern_exit.c:1130 #19 0xffffffff80bbaa2c in kern_wait6 (td=3D0xfffff80a0d384000, = idtype=3DP_ALL,=20 id=3D0, status=3D0xfffffe00db52695c, options=3D48, wrusage=3D0x0, = siginfo=3D0x0) at /usr/src/sys/kern/kern_exit.c:1256 #20 0xffffffff80bba488 in sys_wait4 (td=3D0xfffff80a0d384000,=20 uap=3D0xfffff80a0d3843c8) at /usr/src/sys/kern/kern_exit.c:1171 #21 0xffffffff810e2e7e in amd64_syscall (td=3D0xfffff80a0d384000, = traced=3D0) at src/sys/amd64/amd64/../../kern/subr_syscall.c:135 #22 0xffffffff810bc19d in fast_syscall_common () at /usr/src/sys/amd64/amd64/exception.S:504 #23 0x000000080038f7fa in ?? () Previous frame inner to this frame (corrupt stack?) Current language: auto; currently minimal (kgdb)=20 . . . FreeBSD 13.0-CURRENT #24 r349794M: Sun Jul 7 01:55:57 PDT 2019 = markmi@FBSDFSSD:/usr/obj/amd64_clang/amd64.amd64/usr/src/amd64.amd64/sys/G= ENERIC-NODBG amd64 FreeBSD clang version 8.0.1 (branches/release_80 363030) (based on LLVM = 8.0.1) SRAT: Ignoring memory at addr 0x1428000000 SRAT: Ignoring memory at addr 0x8894000000 VT(vga): text 80x25 Hyper-V Version: 10.0.18362 [SP0] = Features=3D0x2e7f<VPRUNTIME,TMREFCNT,SYNIC,SYNTM,APIC,HYPERCALL,VPINDEX,RE= FTSC,IDLE,TMFREQ> PM Features=3D0x0 [C2] Features3=3D0xbed7b2<DEBUG,XMMHC,IDLE,NUMA,TMFREQ,SYNCMC,CRASH,NPIEP> Timecounter "Hyper-V" frequency 10000000 Hz quality 2000 CPU: AMD Ryzen Threadripper 1950X 16-Core Processor (3393.67-MHz = K8-class CPU) Origin=3D"AuthenticAMD" Id=3D0x800f11 Family=3D0x17 Model=3D0x1 = Stepping=3D1 = Features=3D0x1783fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE= ,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE,SSE2,HTT> = Features2=3D0xfed83203<SSE3,PCLMULQDQ,SSSE3,FMA,CX16,SSE4.1,SSE4.2,MOVBE,P= OPCNT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND,HV> AMD Features=3D0x2e500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM> AMD = Features2=3D0x4003f3<LAHF,CMP,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,Topology> Structured Extended = Features=3D0x209c01a9<FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSH= OPT,SHA> XSAVE Features=3D0xf<XSAVEOPT,XSAVEC,XINUSE,XSAVES> AMD Extended Feature Extensions ID = EBX=3D0x2001004<XSaveErPtr,IBPB,VIRT_SSBD> Hypervisor: Origin =3D "Microsoft Hv" . . . =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5AA725CF-2470-4C79-A329-951675390BF3>