From owner-freebsd-current@FreeBSD.ORG Mon Sep 21 15:36:57 2009 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C05B6106568D for ; Mon, 21 Sep 2009 15:36:57 +0000 (UTC) (envelope-from lab@gta.com) Received: from mailgate.gta.com (mailgate.gta.com [199.120.225.20]) by mx1.freebsd.org (Postfix) with SMTP id 2ABA18FC29 for ; Mon, 21 Sep 2009 15:36:56 +0000 (UTC) Received: (qmail 41291 invoked by uid 1000); 21 Sep 2009 15:36:55 -0000 Date: Mon, 21 Sep 2009 11:36:55 -0400 From: Larry Baird To: freebsd-current@freebsd.org Message-ID: <20090921153655.GA37236@gta.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="OgqxwSJOaUobr8KG" Content-Disposition: inline User-Agent: Mutt/1.4.2.3i Subject: XEN 5.5.0 and clflush X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Sep 2009 15:36:57 -0000 --OgqxwSJOaUobr8KG Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Since the end of August I have been unable to boot a generic kernel from FreeBSD current or 8 under XEN 5.5.0. Finally had a chance to briefly look at the problem. If I apply attached patch to remove calls to clflush() I am able to boot current. Hopefully somebody can shed some light. Is XEN incorrecty reporting CPUID_CLFSH or is XEN not correctly virtualizing this option. Or is the issue someplace else? I have also attached the dmesg from a successful boot. This issue seems to be same as http://www.freebsd.org/cgi/query-pr.cgi?pr=138863 Here is an attempt to type backtrace from non-booting kernel: pmap_invalidate_cache_range(c3252000,c3253000,c3253000,0,fee00000,...) at pamp_invalidate_cache_range+0x60 pmap_mapdev_attr(fee00000,400,0,c1420d34,c0ba7a72,...) at pmap_mapdev_attr+0xec pmap_mapdev() at pmap_mapdev+0x20 lapic_init() at lapic_init+0x32 madt_setup_local() at madt_setup_local+0x2c apic_init() at apic_init+0x11a mistartup() at mi_startup+0x96 begin() at begin+0x2c Larry -- ------------------------------------------------------------------------ Larry Baird | http://www.gta.com Global Technology Associates, Inc. | Orlando, FL Email: lab@gta.com | TEL 407-380-0220, FAX 407-380-6080 --OgqxwSJOaUobr8KG Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="pmap.patch" --- pmap.c.orig 2009-09-21 11:14:53.000000000 -0400 +++ pmap.c 2009-09-21 11:15:01.000000000 -0400 @@ -992,9 +992,10 @@ KASSERT((eva & PAGE_MASK) == 0, ("pmap_invalidate_cache_range: eva not page-aligned")); - if (cpu_feature & CPUID_SS) + if (cpu_feature & CPUID_SS) { ; /* If "Self Snoop" is supported, do nothing. */ - else if (cpu_feature & CPUID_CLFSH) { +#ifdef MAYBE_09_21_2009 + } else if (cpu_feature & CPUID_CLFSH) { /* * Otherwise, do per-cache line flush. Use the mfence @@ -1007,6 +1008,7 @@ for (; sva < eva; sva += cpu_clflush_line_size) clflush(sva); mfence(); +#endif // MAYBE_09_21_2009 } else { /* --OgqxwSJOaUobr8KG Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="dmesg.boot" Copyright (c) 1992-2009 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 9.0-CURRENT #2: Mon Sep 21 11:00:46 EDT 2009 lab@sw-xenoss.gta.com:/usr/src/sys/i386/compile/GENERIC WARNING: WITNESS option enabled, expect reduced performance. Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Xeon(R) CPU 5110 @ 1.60GHz (1691.83-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x6fb Stepping = 11 Features=0x789fbff Features2=0x80002201> AMD Features=0x20000000 AMD Features2=0x1 TSC: P-state invariant real memory = 536870912 (512 MB) avail memory = 493977600 (471 MB) ACPI APIC Table: ioapic0: Changing APIC ID to 1 MADT: Forcing active-low polarity and level trigger for SCI ioapic0 irqs 0-47 on motherboard kbd1 at kbdmux0 acpi0: on motherboard acpi0: [ITHREAD] Timecounter "ACPI-safe" frequency 3579545 Hz quality 850 acpi_timer0: <32-bit timer at 3.579545MHz> port 0x1f48-0x1f4b on acpi0 acpi_hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 62500000 Hz quality 900 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 isab0: at device 1.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xc220-0xc22f at device 1.1 on pci0 ata0: on atapci0 ata0: [ITHREAD] ata1: on atapci0 ata1: [ITHREAD] uhci0: port 0xc200-0xc21f irq 23 at device 1.2 on pci0 uhci0: [ITHREAD] uhci0: LegSup = 0x0000 usbus0: controller did not stop usbus0: on uhci0 pci0: at device 1.3 (no driver attached) vgapci0: mem 0xf0000000-0xf1ffffff,0xf3000000-0xf3000fff at device 2.0 on pci0 pci0: at device 3.0 (no driver attached) re0: port 0xc100-0xc1ff mem 0xf3001000-0xf30010ff irq 32 at device 4.0 on pci0 re0: Chip rev. 0x74800000 re0: MAC rev. 0x00000000 miibus0: on re0 rlphy0: PHY 0 on miibus0 rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto re0: Ethernet address: 4e:c4:60:ce:d0:72 re0: [FILTER] atrtc0: port 0x70-0x71 irq 8 on acpi0 atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] atkbd0: [ITHREAD] psm0: irq 12 on atkbdc0 psm0: [GIANT-LOCKED] psm0: [ITHREAD] psm0: model IntelliMouse Explorer, device ID 4 fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 fdc0: does not respond device_attach: fdc0 attach returned 6 uart0: port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart0: [FILTER] ppc0: port 0x378-0x37f irq 7 on acpi0 ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode ppc0: [ITHREAD] ppbus0: on ppc0 plip0: on ppbus0 plip0: [ITHREAD] lpt0: on ppbus0 lpt0: [ITHREAD] lpt0: Interrupt-driven port ppi0: on ppbus0 cpu0: on acpi0 fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 fdc0: does not respond device_attach: fdc0 attach returned 6 pmtimer0 on isa0 orm0: at iomem 0xd0000-0xd7fff pnpid ORM0000 on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 Timecounter "TSC" frequency 1691834274 Hz quality 800 Timecounters tick every 10.000 msec usbus0: 12Mbps Full Speed USB v1.0 ad0: 20480MB at ata0-master WDMA2 ugen0.1: at usbus0 uhub0: on usbus0 GEOM: ad0s1: geometry does not match label (255h,63s != 16h,63s). uhub0: 2 ports with 2 removable, self powered ugen0.2: at usbus0 ums0: on usbus0 ums0: 3 buttons and [Z] coordinates ID=0 acd0: CDROM at ata1-slave WDMA2 WARNING: WITNESS option enabled, expect reduced performance. Trying to mount root from ufs:/dev/ad0s1a lock order reversal: (sleepable after non-sleepable) 1st 0xc390a058 rtentry (rtentry) @ net/route.c:1409 2nd 0xc0f433b8 ifnet_sx (ifnet_sx) @ netinet/sctp_bsd_addr.c:211 KDB: stack backtrace: db_trace_self_wrapper(c0c828bc,d6957788,c08c2725,c08b354b,c0c85715,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c08b354b,c0c85715,c35286e8,c352c8b8,d69577e4,...) at kdb_backtrace+0x29 _witness_debugger(c0c85715,c0f433b8,c0c8e50e,c352c8b8,c0c9865e,...) at _witness_debugger+0x25 witness_checkorder(c0f433b8,1,c0c98655,d3,0,...) at witness_checkorder+0x839 _sx_slock(c0f433b8,0,c0c98655,d3,d6957878,...) at _sx_slock+0x85 sctp_init_ifns_for_vrf(3,c39018c0,d69578c0,c08c3178,c3901964,...) at sctp_init_ifns_for_vrf+0x30 sctp_addr_change(c38ffa00,1,d69578c0,c08c256c,d695791c,...) at sctp_addr_change+0x2c rt_newaddrmsg(1,c38ffa00,0,c390a000,c38ffa00,...) at rt_newaddrmsg+0x3f rtinit(c38ffa00,1,5,c0de9cfc,51573592,...) at rtinit+0x381 in_ifinit(0,c0c96ee7,1aa,1a6,c3814800,...) at in_ifinit+0x8f6 in_control(c38f7000,8040691a,c3907dc0,c3814800,c39018c0,...) at in_control+0xccb ifioctl(c38f7000,8040691a,c3907dc0,c39018c0,c38ff200,...) at ifioctl+0x14f0 soo_ioctl(c385f540,8040691a,c3907dc0,c356e100,c39018c0,...) at soo_ioctl+0x415 kern_ioctl(c39018c0,3,8040691a,c3907dc0,8bc060,...) at kern_ioctl+0x1fd ioctl(c39018c0,d6957cf8,c,c0c96f81,c0d665c8,...) at ioctl+0x134 syscall(d6957d38) at syscall+0x2a3 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (54, FreeBSD ELF32, ioctl), eip = 0x281bf513, esp = 0xbfbfe61c, ebp = 0xbfbfe658 --- --OgqxwSJOaUobr8KG--