Date: Thu, 1 Sep 2016 15:13:00 +0300 From: Slawa Olhovchenkov <slw@zxy.spb.ru> To: Konstantin Belousov <kostikbel@gmail.com> Cc: stable@freebsd.org, avg@freebsd.org Subject: Re: X2APIC support Message-ID: <20160901121300.GZ88122@zxy.spb.ru> In-Reply-To: <20160901114500.GJ83214@kib.kiev.ua> References: <20151212130615.GE70867@zxy.spb.ru> <20151212133513.GL82577@kib.kiev.ua> <20160901112724.GX88122@zxy.spb.ru> <20160901114500.GJ83214@kib.kiev.ua>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, Sep 01, 2016 at 02:45:00PM +0300, Konstantin Belousov wrote: > On Thu, Sep 01, 2016 at 02:27:24PM +0300, Slawa Olhovchenkov wrote: > > On Sat, Dec 12, 2015 at 03:35:13PM +0200, Konstantin Belousov wrote: > > > > > On Sat, Dec 12, 2015 at 04:06:15PM +0300, Slawa Olhovchenkov wrote: > > > > Does STABLE support X2APIC? > > > > I see X2APIC related commits in CURRENT, what is status for STABLE? > > > > I am try to enable X2APIC support on X10DRi and see kernel trap on > > > > boot. > > > > > > x2APIC is only supported in HEAD. The code to parse processor x2APIC > > > structures in MADT was committed, but AFAIK never tested on real hardware. > > > > > > There are no plans to merge this to stable/10. > > > > I am try now on real hardware and got kernel fault at boot when set in > > BIOS X2APIC_OPT_OUT Flag to [Enable]. > > https://s9.postimg.org/6mt8bu21b/fault2.jpg > How do you know that the issue is due to that flag ? Just by turn on and off this flag and try boot at both cases. > I doubt that BIOS would provide direct knob to manage that flag, since requesting that > out-out only makes sense if BIOS is unable to handle x2APIC mode, e.g. > in SMI handlers. How precisely the BIOS setting called which you frobbed ? As I point before: 'X2APIC_OPT_OUT Flag'. Hidden before 'X2APIC' set to '[Enable]' https://s15.postimg.org/3k5l7z397/bios_x2.jpg > Show complete verbose dmesg from the first line to backtrace. Booting... \01;00Table 'FACP' at 0x79b189a8 Table 'APIC' at 0x79b18ab8 APIC: Found table at 0x79b18ab8 APIC: Using the MADT enumerator. MADT: Found CPU APIC ID 0 ACPI ID 0: enabled SMP: Added CPU 0 (AP) MADT: Found CPU APIC ID 2 ACPI ID 2: enabled SMP: Added CPU 2 (AP) MADT: Found CPU APIC ID 4 ACPI ID 4: enabled SMP: Added CPU 4 (AP) MADT: Found CPU APIC ID 6 ACPI ID 6: enabled SMP: Added CPU 6 (AP) MADT: Found CPU APIC ID 8 ACPI ID 8: enabled SMP: Added CPU 8 (AP) MADT: Found CPU APIC ID 10 ACPI ID 10: enabled SMP: Added CPU 10 (AP) MADT: Found CPU APIC ID 16 ACPI ID 16: enabled SMP: Added CPU 16 (AP) MADT: Found CPU APIC ID 18 ACPI ID 18: enabled SMP: Added CPU 18 (AP) MADT: Found CPU APIC ID 20 ACPI ID 20: enabled SMP: Added CPU 20 (AP) MADT: Found CPU APIC ID 22 ACPI ID 22: enabled SMP: Added CPU 22 (AP) MADT: Found CPU APIC ID 24 ACPI ID 24: enabled SMP: Added CPU 24 (AP) MADT: Found CPU APIC ID 26 ACPI ID 26: enabled SMP: Added CPU 26 (AP) MADT: Found CPU APIC ID 32 ACPI ID 32: enabled SMP: Added CPU 32 (AP) MADT: Found CPU APIC ID 34 ACPI ID 34: enabled SMP: Added CPU 34 (AP) MADT: Found CPU APIC ID 36 ACPI ID 36: enabled SMP: Added CPU 36 (AP) MADT: Found CPU APIC ID 38 ACPI ID 38: enabled SMP: Added CPU 38 (AP) MADT: Found CPU APIC ID 40 ACPI ID 40: enabled SMP: Added CPU 40 (AP) MADT: Found CPU APIC ID 42 ACPI ID 42: enabled SMP: Added CPU 42 (AP) MADT: Found CPU APIC ID 48 ACPI ID 48: enabled SMP: Added CPU 48 (AP) MADT: Found CPU APIC ID 50 ACPI ID 50: enabled SMP: Added CPU 50 (AP) MADT: Found CPU APIC ID 52 ACPI ID 52: enabled SMP: Added CPU 52 (AP) MADT: Found CPU APIC ID 54 ACPI ID 54: enabled SMP: Added CPU 54 (AP) MADT: Found CPU APIC ID 56 ACPI ID 56: enabled SMP: Added CPU 56 (AP) MADT: Found CPU APIC ID 58 ACPI ID 58: enabled SMP: Added CPU 58 (AP) Copyright (c) 1992-2016 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 11.0-RELEASE-p305117 #0: Wed Aug 31 17:58:35 UTC 2016 slw@storage01.int.integros.com:/usr/obj/usr/src/sys/VSTREAM amd64 FreeBSD clang version 3.8.0 (tags/RELEASE_380/final 262564) (based on LLVM 3.8.0) Table 'FACP' at 0x79b189a8 Table 'APIC' at 0x79b18ab8 Table 'FPDT' at 0x79b18c70 Table 'FIDT' at 0x79b18cb8 Table 'SPMI' at 0x79b18d58 Table 'MCFG' at 0x79b18da0 Table 'UEFI' at 0x79b18de0 Table 'MCEJ' at 0x79b18e28 Table 'HPET' at 0x79b18f58 Table 'WDDT' at 0x79b18f90 Table 'SSDT' at 0x79b18fd0 Table 'SSDT' at 0x79b30020 Table 'SSDT' at 0x79b32670 Table 'PRAD' at 0x79b326d8 Table 'DMAR' at 0x79b32798 Table 'HEST' at 0x79b328e8 Table 'BERT' at 0x79b32990 Table 'ERST' at 0x79b329c0 Table 'EINJ' at 0x79b32bf0 ACPI: No SRAT table found PPIM 0: PA=0xb8000, VA=0xffffffff81610000, size=0x8000, mode=0 VT(vga): text 80x25 Preloaded elf kernel "/boot/kernel.VSTREAM/kernel" at 0xffffffff814a8000. Preloaded /boot/entropy "/boot/entropy" at 0xffffffff814a8ac0. Preloaded elf obj module "/boot/kernel.VSTREAM/zfs.ko" at 0xffffffff814a8b10. Preloaded elf obj module "/boot/kernel.VSTREAM/krpc.ko" at 0xffffffff814a9300. Preloaded elf obj module "/boot/kernel.VSTREAM/opensolaris.ko" at 0xffffffff814a9970. Preloaded elf obj module "/boot/kernel.VSTREAM/if_igb.ko" at 0xffffffff814a9fa8. Preloaded elf obj module "/boot/kernel.VSTREAM/if_lagg.ko" at 0xffffffff814aa618. Preloaded elf obj module "/boot/kernel.VSTREAM/ukbd.ko" at 0xffffffff814aacc8. Preloaded elf obj module "/boot/kernel.VSTREAM/usb.ko" at 0xffffffff814ab378. Preloaded elf obj module "/boot/kernel.VSTREAM/umass.ko" at 0xffffffff814abaa8. Preloaded elf obj module "/boot/kernel.VSTREAM/accf_http.ko" at 0xffffffff814ac098. Preloaded elf obj module "/boot/kernel.VSTREAM/cc_htcp.ko" at 0xffffffff814ac610. Preloaded elf obj module "/boot/kernel.VSTREAM/uhci.ko" at 0xffffffff814acc40. Preloaded elf obj module "/boot/kernel.VSTREAM/ohci.ko" at 0xffffffff814ad1f0. Preloaded elf obj module "/boot/kernel.VSTREAM/ehci.ko" at 0xffffffff814ad7a0. Preloaded elf obj module "/boot/kernel.VSTREAM/xhci.ko" at 0xffffffff814add50. Calibrating TSC clock ... TSC clock: 2200043851 Hz CPU: Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz (2200.04-MHz K8-class CPU) Origin="GenuineIntel" Id=0x406f1 Family=0x6 Model=0x4f Stepping=1 Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE> Features2=0x7ffefbff<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND> AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM> AMD Features2=0x121<LAHF,ABM,Prefetch> Structured Extended Features=0x21cbfbb<FSGSBASE,TSCADJ,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,NFPUSG,PQE,RDSEED,ADX,SMAP,PROCTRACE> XSAVE Features=0x1<XSAVEOPT> VT-x: Basic Features=0xda0400<SMM,INS/OUTS,TRUE> Pin-Based Controls=0xff<ExtINT,NMI,VNMI,PreTmr,PostIntr> Primary Processor Controls=0xfff9fffe<INTWIN,TSCOff,HLT,INVLPG,MWAIT,RDPMC,RDTSC,CR3-LD,CR3-ST,CR8-LD,CR8-ST,TPR,NMIWIN,MOV-DR,IO,IOmap,MTF,MSRmap,MONITOR,PAUSE> Secondary Processor Controls=0x77fff<APIC,EPT,DT,RDTSCP,x2APIC,VPID,WBINVD,UG,APIC-reg,VID,PAUSE-loop,RDRAND,INVPCID,VMFUNC,VMCS,XSAVES> Exit Controls=0xda0400<PAT-LD,EFER-SV,PTMR-SV> Entry Controls=0xda0400 EPT Features=0x6334141<XO,PW4,UC,WB,2M,1G,INVEPT,AD,single,all> VPID Features=0xf01<INVVPID,individual,single,all,single-globals> TSC: P-state invariant, performance statistics Data TLB: 2 MByte or 4 MByte pages, 4-way set associative, 32 entries and a separate array with 1 GByte pages, 4-way set associative, 4 entries Data TLB: 4 KB pages, 4-way set associative, 64 entries Instruction TLB: 2M/4M pages, fully associative, 8 entries Instruction TLB: 4KByte pages, 8-way set associative, 128 entries 64-Byte prefetching Shared 2nd-Level TLB: 4 KByte /2 MByte pages, 6-way associative, 1536 entries. Also 1GBbyte pages, 4-way, 16 entries L2 cache: 256 kbytes, 8-way associative, 64 bytes/line real memory = 137438953472 (131072 MB) Physical memory chunk(s): 0x0000000000010000 - 0x0000000000095fff, 548864 bytes (134 pages) 0x0000000000100000 - 0x00000000001fffff, 1048576 bytes (256 pages) 0x00000000014e9000 - 0x000000007918bfff, 2009739264 bytes (490659 pages) 0x0000000100000000 - 0x0000001fabf78fff, 131734147072 bytes (32161657 pages) avail memory = 133408907264 (127228 MB) Table 'FACP' at 0x79b189a8 Table 'APIC' at 0x79b18ab8 Table 'FPDT' at 0x79b18c70 Table 'FIDT' at 0x79b18cb8 Table 'SPMI' at 0x79b18d58 Table 'MCFG' at 0x79b18da0 Table 'UEFI' at 0x79b18de0 Table 'MCEJ' at 0x79b18e28 Table 'HPET' at 0x79b18f58 Table 'WDDT' at 0x79b18f90 Table 'SSDT' at 0x79b18fd0 Table 'SSDT' at 0x79b30020 Table 'SSDT' at 0x79b32670 Table 'PRAD' at 0x79b326d8 Table 'DMAR' at 0x79b32798 DMAR: Found table at 0x79b32798 x2APIC available but disabled by DMAR table Event timer "LAPIC" quality 600 LAPIC: ipi_wait() us multiplier 1 (r 116268019 tsc 2200043851) ACPI APIC Table: <ALASKA A M I > Package ID shift: 5 L3 cache ID shift: 5 L2 cache ID shift: 1 L1 cache ID shift: 1 Core ID shift: 1 kernel trap 12 with interrupts disabled Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = ff fault virtual address = 0x0 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff80537e74 stack pointer = 0x28:0xffffffff814b4a60 frame pointer = 0x28:0xffffffff814b4a70 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = resume, IOPL = 0 current process = 0 () trap number = 12 panic: page fault cpuid = 0 KDB: stack backtrace: #0 0xffffffff805272e7 at kdb_backtrace+0x67 #1 0xffffffff804dd662 at vpanic+0x182 #2 0xffffffff804dd4d3 at panic+0x43 #3 0xffffffff807a3791 at trap_fatal+0x351 #4 0xffffffff807a3983 at trap_pfault+0x1e3 #5 0xffffffff807a2f0c at trap+0x26c #6 0xffffffff80787ca1 at calltrap+0x8 #7 0xffffffff8083b52a at topo_probe+0x61a #8 0xffffffff8078fe81 at cpu_mp_start+0x1b1 #9 0xffffffff805382ca at mp_start+0x3a #10 0xffffffff80465cd8 at mi_startup+0x118 #11 0xffffffff8028dfac at btext+0x2c Uptime: 1s > > > > 0xffffffff80537e74 is /usr/src/sys/kern/subr_smp.c:959 > And this is a topology probe code, which should not be affected by the > x2APIC mode at all, except possibly due to longer format of the APIC IDs. > > I added Andrey to Cc:, he might give some more guidance. > > > > Boot success if X2APIC_OPT_OUT Flag set to [Disable] > > Supermicro X10DRi, dual E5-2650v4
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20160901121300.GZ88122>