From owner-freebsd-xen@FreeBSD.ORG Mon Mar 25 11:06:54 2013 Return-Path: Delivered-To: freebsd-xen@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id AE239F0 for ; Mon, 25 Mar 2013 11:06:54 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 8F739E3 for ; Mon, 25 Mar 2013 11:06:54 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.6/8.14.6) with ESMTP id r2PB6sj6007383 for ; Mon, 25 Mar 2013 11:06:54 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.6/8.14.6/Submit) id r2PB6sTY007381 for freebsd-xen@FreeBSD.org; Mon, 25 Mar 2013 11:06:54 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 25 Mar 2013 11:06:54 GMT Message-Id: <201303251106.r2PB6sTY007381@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-xen@FreeBSD.org Subject: Current problem reports assigned to freebsd-xen@FreeBSD.org X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Mar 2013 11:06:54 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/176471 xen [xen] xn driver crash on detach o kern/176053 xen [xen] [patch] i386: Correct wrong usage of vsnprintf() o kern/175954 xen [xen] XENHVM xn network driver extreme packet loss dur o kern/175822 xen [xen] FreeBSD 9.1 does not work with Xen 4.0 o kern/175757 xen [xen] [patch] xen pvhvm looses keyboard input from VNC o kern/171873 xen [xen] xn network device floods warning in dmesg o kern/171118 xen [xen] FreeBSD XENHVM guest doesn't shutdown cleanly o kern/166174 xen [xen] Problems ROOT MOUNT ERROR o kern/165418 xen [xen] Problems mounting root filesystem from XENHVM o kern/164630 xen [xen] XEN HVM kernel: run_interrupt_driven_hooks: stil o kern/164450 xen [xen] Failed to install FreeeBSD 9.0-RELEASE from CD i o kern/162677 xen [xen] FreeBSD not compatible with "Current Stable Xen" o kern/161318 xen [xen] sysinstall crashes with floating point exception o kern/155468 xen [xen] Xen PV i386 multi-kernel CPU system is not worki o kern/155353 xen [xen] [patch] put "nudging TOD" message under boot_ver o kern/154833 xen [xen]: xen 4.0 - DomU freebsd8.2RC3 i386, XEN kernel. o kern/154473 xen [xen] xen 4.0 - DomU freebsd8.1 i386, XEN kernel. Not o kern/154472 xen [xen] xen 4.0 - DomU freebsd8.1 i386 xen kernel reboot o kern/154428 xen [xen] xn0 network interface and PF - Massive performan o kern/153674 xen [xen] i386/XEN idle thread shows wrong percentages o kern/153672 xen [xen] [panic] i386/XEN panics under heavy fork load o kern/153620 xen [xen] Xen guest system clock drifts in AWS EC2 (FreeBS o kern/153477 xen [xen] XEN pmap code abuses vm page queue lock o kern/153150 xen [xen] xen/ec2: disable checksum offloading on interfac o kern/152228 xen [xen] [panic] Xen/PV panic with machdep.idle_mwait=1 o kern/144629 xen [xen] FreeBSD 8-RELEASE XEN pvm networking doesn't wor o kern/143398 xen [xen] FreeBSD 8-RELEASE XEN pvm networking doesn't wor o kern/143340 xen [xen] FreeBSD 8-RELEASE XEN pvm networking doesn't wor f kern/143069 xen [xen] [panic] Xen Kernel Panic - Memory modified after f kern/135667 xen ufs filesystem corruption on XEN DomU system f kern/135421 xen [xen] FreeBSD Xen PVM DomU network failure - netfronc. f kern/135178 xen [xen] Xen domU outgoing data transfer stall when TSO i p kern/135069 xen [xen] FreeBSD-current/Xen SMP doesn't function at all f i386/124516 xen [xen] FreeBSD-CURRENT Xen Kernel Segfaults when config o kern/118734 xen [xen] FreeBSD 6.3-RC1 and FreeBSD 7.0-BETA 4 fail to b 35 problems total. From owner-freebsd-xen@FreeBSD.ORG Tue Mar 26 09:32:05 2013 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 8E8AD262 for ; Tue, 26 Mar 2013 09:32:05 +0000 (UTC) (envelope-from roger.pau@citrix.com) Received: from SMTP.EU.CITRIX.COM (smtp.eu.citrix.com [46.33.159.39]) by mx1.freebsd.org (Postfix) with ESMTP id B2196B28 for ; Tue, 26 Mar 2013 09:32:04 +0000 (UTC) X-IronPort-AV: E=Sophos;i="4.84,911,1355097600"; d="scan'208";a="2907556" Received: from lonpmailmx01.citrite.net ([10.30.203.162]) by LONPIPO01.EU.CITRIX.COM with ESMTP/TLS/RC4-MD5; 26 Mar 2013 09:31:52 +0000 Received: from [192.168.1.30] (10.30.249.104) by LONPMAILMX01.citrite.net (10.30.203.162) with Microsoft SMTP Server id 8.3.298.1; Tue, 26 Mar 2013 09:31:51 +0000 Message-ID: <51516B06.8080704@citrix.com> Date: Tue, 26 Mar 2013 10:31:50 +0100 From: =?ISO-8859-1?Q?Roger_Pau_Monn=E9?= User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:17.0) Gecko/20130307 Thunderbird/17.0.4 MIME-Version: 1.0 To: tech mailinglists Subject: Re: [Xen-users] Trouble booting FreeBSD i386 PV DomU References: In-Reply-To: Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 7bit Cc: xen-users , "freebsd-xen@freebsd.org" X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Mar 2013 09:32:05 -0000 On 21/02/13 10:18, tech mailinglists wrote: > Hello all, > > I have created a FreeBSD PV DomU image formatted with ZFS. I compiled > FreeBSD with KERNCONF=XEN for the kernel and the normal world and > distribution target. Then I transfered it to a Debian Dom0 with Xen 4.2.1. > > I tried to boot the image befor I migrate it to LVM and I now get teh > following output: > > Parsing config from freebsd-test.cfg > libxl: error: libxl_dm.c:1212:device_model_spawn_outcome: domain 12 > device model: spawn failed (rc=-3) > libxl: error: libxl_qmp.c:641:libxl__qmp_initialize: Connection error: > No such file or directory > Daemon running with PID 3451 > WARNING: loader(8) metadata is missing! > GDB: no debug ports present > KDB: debugger backends: ddb > KDB: current backend: ddb > APIC: Using the MPTable enumerator. > SMP: Added CPU 0 (BSP) > Copyright (c) 1992-2012 The FreeBSD Project. > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 > The Regents of the University of California. All rights reserved. > FreeBSD is a registered trademark of The FreeBSD Foundation. > FreeBSD 9.1-RELEASE #0: Wed Feb 20 14:16:03 CET 2013 > root@build:/usr/obj/usr/src/sys/XEN i386 > WARNING: WITNESS option enabled, expect reduced performance. > Xen reported: 3341.754 MHz processor. > Timecounter "ixen" frequency 1953125 Hz quality 0 > CPU: Intel(R) Core(TM) i7 CPU 975 @ 3.33GHz (3341.75-MHz > 686-class CPU) > Origin = "GenuineIntel" Id = 0x106a5 Family = 6 Model = 1a > Stepping = 5 > > Features=0xbfe3fbff > > Features2=0x98e3bd > AMD Features=0x28100000 > AMD Features2=0x1 > > Data TLB: 4 KB pages, 4-way set associative, 64 entries > 1st-level data cache: 32 KB, 8-way set associative, 64 byte line size > L2 cache: 256 kbytes, 8-way associative, 64 bytes/line > real memory = 2147483648 (2048 MB) > Physical memory chunk(s): > 0x0000000000a58000 - 0x000000007d995fff, 2096357376 bytes (511806 pages) > avail memory = 2092105728 (1995 MB) > INTR: Adding local APIC 0 as a target > ULE: setup cpu 0 > [XEN] IPI cpu=0 irq=128 vector=RESCHEDULE_VECTOR (0) > [XEN] IPI cpu=0 irq=129 vector=CALL_FUNCTION_VECTOR (1) > Event-channel device installed. > io: > random: > mem: > Pentium Pro MTRR support enabled > null: > nfslock: pseudo-device > [XEN] xen_rtc_probe: probing Hypervisor RTC clock > rtc0: on motherboard > [XEN] xen_rtc_attach: attaching Hypervisor RTC clock > rtc0: registered as a time-of-day clock (resolution 1000000us, > adjustment 0.500000000s) > xenstore0: on motherboard > Grant table initialized > xc0: on motherboard > Device configuration finished. > procfs registered > Event timer "ixen" quality 600 > Timecounters tick every 10.000 msec > lo0: bpf attached > xenbusb_front0: on xenstore0 > xenbusb_add_device: Device device/suspend/event-channel ignored. State 6 > xenbusb_back0: on xenstore0 > xctrl0: on xenstore0 > [XEN] hypervisor wallclock nudged; nudging TOD. > run_interrupt_driven_hooks: still waiting after 60 seconds for > xenbus_free_evtchn > run_interrupt_driven_hooks: still waiting after 120 seconds for > xenbus_free_evtchn > run_interrupt_driven_hooks: still waiting after 180 seconds for > xenbus_free_evtchn > run_interrupt_driven_hooks: still waiting after 240 seconds for > xenbus_free_evtchn > run_interrupt_driven_hooks: still waiting after 300 seconds for > xenbus_free_evtchn > panic: run_interrupt_driven_config_hooks: waited too long > cpuid = 0 > KDB: enter: panic > [ thread pid 0 tid 100000 ] > Stopped at kdb_enter+0x3a: movl $0,kdb_why > db> Hello, I've been trying to get a FreeBSD PV guest, I've installed FreeBSD i386 HEAD and then tried to recompile the kernel using KERNCONF=XEN, but Xen refuses to load the resulting kernel: root@loki:~# xl -vvv create -c freebsd32pv.cfg Parsing config from freebsd32pv.cfg libxl: debug: libxl_create.c:1174:do_domain_create: ao 0x22e5a20: create: how=(nil) callback=(nil) poller=0x22e5a80 libxl: debug: libxl_device.c:229:libxl__device_disk_set_backend: Disk vdev=hda spec.backend=unknown libxl: debug: libxl_device.c:265:libxl__device_disk_set_backend: Disk vdev=hda, using backend phy libxl: debug: libxl_create.c:677:initiate_domain_create: running bootloader libxl: debug: libxl_bootloader.c:327:libxl__bootloader_run: no bootloader configured, using user supplied kernel libxl: debug: libxl_event.c:561:libxl__ev_xswatch_deregister: watch w=0x22e60c0: deregister unregistered libxl: debug: libxl_numa.c:435:libxl__get_numa_candidate: New best NUMA placement candidate found: nr_nodes=1, nr_cpus=8, nr_vcpus=15, free_memkb=4353 libxl: detail: libxl_dom.c:192:numa_place_domain: NUMA placement candidate with 1 nodes, 8 cpus and 4353 KB free selected domainbuilder: detail: xc_dom_allocate: cmdline="boot_verbose=1,vfs.root.mountfrom=ufs:/dev/ad0s1a,kern.hz=100", features="(null)" libxl: debug: libxl_dom.c:380:libxl__build_pv: pv kernel mapped 0 path /root/kernel.freebsd domainbuilder: detail: xc_dom_kernel_file: filename="/root/kernel.freebsd" domainbuilder: detail: xc_dom_malloc_filemap : 5235 kB domainbuilder: detail: xc_dom_boot_xen_init: ver 4.2, caps xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 domainbuilder: detail: xc_dom_parse_image: called domainbuilder: detail: xc_dom_find_loader: trying ELF-generic loader ... domainbuilder: detail: loader probe OK xc: detail: elf_parse_binary: phdr: paddr=0xc0000000 memsz=0x439eb8 xc: detail: elf_parse_binary: phdr: paddr=0xc043a000 memsz=0x23a578 xc: detail: elf_parse_binary: memory: 0xc0000000 -> 0xc0674578 xc: detail: elf_xen_parse: __xen_guest: "LOADER=generic,GUEST_OS=freebsd,GUEST_VER=7.0,XEN_VER=xen-3.0,BSD_SYMTAB,VIRT_BASE=0xc0000000" xc: detail: elf_xen_parse_guest_info: LOADER="generic" xc: detail: elf_xen_parse_guest_info: GUEST_OS="freebsd" xc: detail: elf_xen_parse_guest_info: GUEST_VER="7.0" xc: detail: elf_xen_parse_guest_info: XEN_VER="xen-3.0" xc: detail: elf_xen_parse_guest_info: BSD_SYMTAB="" xc: detail: elf_xen_parse_guest_info: VIRT_BASE="0xc0000000" xc: detail: elf_xen_addr_calc_check: ELF_PADDR_OFFSET unset, using 0xc0000000 xc: detail: elf_xen_addr_calc_check: addresses: xc: detail: virt_base = 0xc0000000 xc: detail: elf_paddr_offset = 0xc0000000 xc: detail: virt_offset = 0x0 xc: detail: virt_kstart = 0xc0000000 xc: detail: virt_kend = 0xc07439d0 xc: detail: virt_entry = 0xc0046000 xc: detail: p2m_base = 0xffffffffffffffff domainbuilder: detail: xc_dom_load_elf_symtab/parse: bsd_symtab_start=c07439d0, kernel.end=0xc07439d0 -- symtab=0xc07439d4, maxaddr=0xc0743e40 domainbuilder: detail: xc_dom_load_elf_symtab: h=25 symtab, size=0x4a310, maxaddr=0xc078e150 domainbuilder: detail: xc_dom_load_elf_symtab: h=26 strtab, size=0x69500, maxaddr=0xc07f7650 domainbuilder: detail: xc_dom_parse_elf_kernel: xen-3.0-x86_32: 0xc0000000 -> 0xc07f7650 domainbuilder: detail: xc_dom_mem_init: mem 512 MB, pages 0x20000 pages, 4k each domainbuilder: detail: xc_dom_mem_init: 0x20000 pages domainbuilder: detail: xc_dom_boot_mem_init: called domainbuilder: detail: xc_dom_malloc : 1024 kB domainbuilder: detail: xc_dom_build_image: called domainbuilder: detail: xc_dom_alloc_segment: kernel : 0xc0000000 -> 0xc07f8000 (pfn 0x0 + 0x7f8 pages) domainbuilder: detail: xc_dom_pfn_to_ptr: domU mapping: pfn 0x0+0x7f8 at 0x7f88fc22c000 xc: detail: elf_load_binary: phdr 2 at 0x0x7f88fc22c000 -> 0x0x7f88fc665eb8 xc: detail: elf_load_binary: phdr 3 at 0x0x7f88fc666000 -> 0x0x7f88fc694c9c xc: detail: elf_load_bsdsyms: shdr 4 at 0x0x7f88fca4bd84 -> 0x0x7f88fc8a09e8 xc: detail: elf_load_bsdsyms: shdr 24 at 0x0x7f88fce8cec4 -> 0x0x7f88fc8bc0a4 xc: detail: elf_load_bsdsyms: shdr 25 at 0x0x7f88fce8d418 -> 0x0x7f88fc8bc1c0 xc: detail: elf_load_bsdsyms: shdr 26 at 0x0x7f88fced7728 -> 0x0x7f88fc9064d0 domainbuilder: detail: xc_dom_load_elf_symtab/load: bsd_symtab_start=c07439d0, kernel.end=0xc07f8000 -- symtab=0xc07439d4, maxaddr=0xc0743e40 domainbuilder: detail: xc_dom_load_elf_symtab: h=25 symtab, size=0x4a310, maxaddr=0xc078e150 domainbuilder: detail: xc_dom_load_elf_symtab: h=26 strtab, size=0x69500, maxaddr=0xc07f7650 domainbuilder: detail: xc_dom_alloc_segment: phys2mach : 0xc07f8000 -> 0xc0878000 (pfn 0x7f8 + 0x80 pages) domainbuilder: detail: xc_dom_pfn_to_ptr: domU mapping: pfn 0x7f8+0x80 at 0x7f88fec35000 domainbuilder: detail: xc_dom_alloc_page : start info : 0xc0878000 (pfn 0x878) domainbuilder: detail: xc_dom_alloc_page : xenstore : 0xc0879000 (pfn 0x879) domainbuilder: detail: xc_dom_alloc_page : console : 0xc087a000 (pfn 0x87a) domainbuilder: detail: nr_page_tables: 0x00000000ffffffff/32: 0x0000000000000000 -> 0x00000000ffffffff, 1 table(s) domainbuilder: detail: nr_page_tables: 0x00000000003fffff/22: 0x00000000c0000000 -> 0x00000000c0bfffff, 3 table(s) domainbuilder: detail: xc_dom_alloc_segment: page tables : 0xc087b000 -> 0xc087f000 (pfn 0x87b + 0x4 pages) domainbuilder: detail: xc_dom_pfn_to_ptr: domU mapping: pfn 0x87b+0x4 at 0x7f88fedbe000 domainbuilder: detail: xc_dom_alloc_page : boot stack : 0xc087f000 (pfn 0x87f) domainbuilder: detail: xc_dom_build_image : virt_alloc_end : 0xc0880000 domainbuilder: detail: xc_dom_build_image : virt_pgtab_end : 0xc0c00000 domainbuilder: detail: xc_dom_boot_image: called domainbuilder: detail: arch_setup_bootearly: doing nothing domainbuilder: detail: xc_dom_compat_check: supported guest type: xen-3.0-x86_64 domainbuilder: detail: xc_dom_compat_check: supported guest type: xen-3.0-x86_32p domainbuilder: detail: xc_dom_compat_check: supported guest type: hvm-3.0-x86_32 domainbuilder: detail: xc_dom_compat_check: supported guest type: hvm-3.0-x86_32p domainbuilder: detail: xc_dom_compat_check: supported guest type: hvm-3.0-x86_64 xc: error: panic: xc_dom_boot.c:127: xc_dom_compat_check: guest type xen-3.0-x86_32 not supported by xen kernel, sorry: Invalid kernel It refuses to boot because it detects the kernel doesn't support PAE. I've hacked sys/i386/xen/locore.s in order to add the PAE flag: .ascii "LOADER=generic,GUEST_OS=freebsd,GUEST_VER=7.0,XEN_VER=xen-3.0,BSD_SYMTAB,VIRT_BASE=0xc0000000,PAE=yes[extended-cr3]" I know there's an elfnote that should tell Xen if kernel support PAE or not, but it seems to be ignored: ELFNOTE(Xen, XEN_ELFNOTE_PAE_MODE, .asciz, "yes") Anyway, I've recompiled the kernel with the PAE flag hack, but then when trying to load the resulting kernel it crashes very early: (XEN) d12:v0: unhandled page fault (ec=0002) (XEN) Pagetable walk from 0000000000000100: (XEN) L4[0x000] = 0000000141710027 000000000000087b (XEN) L3[0x000] = 0000000000000000 ffffffffffffffff (XEN) domain_crash_sync called from entry.S (XEN) Domain 12 (vcpu#0) crashed on cpu#7: (XEN) ----[ Xen-4.2.1 x86_64 debug=n Not tainted ]---- (XEN) CPU: 7 (XEN) RIP: e019:[<00000000c00452a0>] (XEN) RFLAGS: 0000000000000246 EM: 1 CONTEXT: pv guest (XEN) rax: 0000000000000100 rbx: 0000000000000000 rcx: 0000000000000000 (XEN) rdx: 00000000c0878000 rsi: 0000000000000000 rdi: 0000000000000000 (XEN) rbp: 00000000c043e12c rsp: 00000000c043e07c r8: 0000000000000000 (XEN) r9: 0000000000000000 r10: 0000000000000000 r11: 0000000000000000 (XEN) r12: 0000000000000000 r13: 0000000000000000 r14: 0000000000000000 (XEN) r15: 0000000000000000 cr0: 000000008005003b cr4: 00000000000026f0 (XEN) cr3: 000000013fb58000 cr2: 0000000000000100 (XEN) ds: e021 es: e021 fs: e021 gs: e021 ss: e021 cs: e019 (XEN) Guest stack trace from esp=c043e07c: (XEN) 00000002 c00452a0 0001e019 00010046 c03adc1f 00000000 00000000 00000000 (XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 (XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 (XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 (XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 (XEN) 00000000 c0878000 00000000 c0884000 00000000 c0046015 c0878000 c043e138 (XEN) 0000e021 00000040 00000020 c0468d0c 01800000 00000001 c014ffe0 c043e2c4 (XEN) 01800000 0fffffff c01500b0 c043e2c4 01b00000 01000000 c016ef90 c043e2d4 (XEN) 01b00000 01000000 c016f0b0 c0647994 c0647a3c 00000000 ffffffff c0000001 (XEN) c0469810 00000000 c03db795 00000000 c03ecf36 00000000 00000000 c03db81e (XEN) c0469810 00000000 ffffffff 80040006 c0469804 00000000 c03db7f3 c0172990 (XEN) c03db706 00000000 00000000 c03db7fa c0469810 00000000 ffffffff 80040006 (XEN) c043e2c0 00000000 c03db786 c0172990 c03db706 00000000 00000000 c03db7c7 (XEN) c0469810 00000000 ffffffff 80040002 c04697fc 00000000 c03db79d c0172990 (XEN) c03de14d 00000000 00000000 c03db7a8 02380000 0fffffff c004a6b0 00000000 (XEN) c0469810 00000000 ffffffff c0000006 00000000 00000000 c03db643 c004a520 (XEN) c03db706 00000000 00000000 c03db709 c0469810 00000000 ffffffff 80000003 (XEN) 00000000 00000000 c03fb8f6 c004a470 c0433acf 00000000 00000000 c03db6d7 (XEN) 0000c000 00500000 00000000 3454f8e9 c03db791 00000000 c0647994 c03db832 (XEN) 00000000 c0436966 c004baa0 00000000 00000000 00000000 00000000 c0408707 Is Xen i386 PV broken? Or I'm being incredebly stupid and missing something obvious? From owner-freebsd-xen@FreeBSD.ORG Tue Mar 26 09:40:14 2013 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id C81604A7 for ; Tue, 26 Mar 2013 09:40:14 +0000 (UTC) (envelope-from bounces+73574-0602-freebsd-xen=freebsd.org@sendgrid.me) Received: from o3.shared.sendgrid.net (o3.shared.sendgrid.net [208.117.48.85]) by mx1.freebsd.org (Postfix) with SMTP id 75324CEF for ; Tue, 26 Mar 2013 09:40:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sendgrid.info; h=from :mime-version:to:cc:subject:references:in-reply-to:content-type :content-transfer-encoding; s=smtpapi; bh=EFUDOoixBAwlQLq3+h2la7 WxrQY=; b=CUGu6WSrk6jsTe8/haSznhauxW2/qskv3V0rPzvXdOhcOim0m5tA3r eTLW3jvfqjIGxsm+VMYNznxfp6suLp0Jqlz2/jjKwifhi60lmKE/vFaf82abkrs9 GHMBFe+FNFXEO4xr6MNGDulmENp9Jo/zdfIWPOkTQZt23/ksi0BNo= Received: by 10.4.35.216 with SMTP id mf7.7854.51516CFD6 Tue, 26 Mar 2013 09:40:13 +0000 (UTC) Received: from mail.tarsnap.com (unknown [10.60.208.17]) by mi17 (SG) with ESMTP id 51516cfd.6102.1459523 for ; Tue, 26 Mar 2013 04:40:13 -0500 (CST) Received: (qmail 67646 invoked from network); 26 Mar 2013 09:40:12 -0000 Received: from unknown (HELO clamshell.daemonology.net) (127.0.0.1) by ec2-107-20-205-189.compute-1.amazonaws.com with ESMTP; 26 Mar 2013 09:40:12 -0000 Received: (qmail 16134 invoked from network); 26 Mar 2013 09:38:50 -0000 Received: from unknown (HELO clamshell.daemonology.net) (127.0.0.1) by clamshell.daemonology.net with SMTP; 26 Mar 2013 09:38:50 -0000 Message-ID: <51516CAA.3040802@freebsd.org> Date: Tue, 26 Mar 2013 02:38:50 -0700 From: Colin Percival User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130314 Thunderbird/17.0.4 MIME-Version: 1.0 To: =?ISO-8859-1?Q?Roger_Pau_Monn=E9?= Subject: Re: [Xen-users] Trouble booting FreeBSD i386 PV DomU References: <51516B06.8080704@citrix.com> In-Reply-To: <51516B06.8080704@citrix.com> X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=ISO-8859-1 content-transfer-encoding: quoted-printable X-SG-EID: RUbAm5H8PjswBj/QH+sYVehaJogg3iBnZcyVi1bw/IyfhEHCo1ffAUKuzc5jZ4woeeKZs5gdNudyS/NNgbOgshLVsBB25dLrnLM2zPt2/jv6aKRQthqmOWMqjACu+vJX Cc: xen-users , "freebsd-xen@freebsd.org" X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Mar 2013 09:40:14 -0000 On 03/26/13 02:31, Roger Pau Monn=E9 wrote:=0D > Is Xen i386 PV broken?=0D =0D Not completely broken, but it's certainly not in a good state. I believe= =0D it's broken with SMP, for example -- if the "crashed on cpu#7" in your=0D output means cpu#7 from the guest, it would certainly explain things.=0D =0D HVM is the way to go with FreeBSD/Xen.=0D =0D -- =0D Colin Percival=0D Security Officer Emeritus, FreeBSD | The power to serve=0D Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid= =0D From owner-freebsd-xen@FreeBSD.ORG Tue Mar 26 10:10:19 2013 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 0F83DEA9; Tue, 26 Mar 2013 10:10:19 +0000 (UTC) (envelope-from roger.pau@citrix.com) Received: from SMTP.EU.CITRIX.COM (smtp.eu.citrix.com [46.33.159.39]) by mx1.freebsd.org (Postfix) with ESMTP id 2A96AF63; Tue, 26 Mar 2013 10:10:17 +0000 (UTC) X-IronPort-AV: E=Sophos;i="4.84,911,1355097600"; d="scan'208";a="2909776" Received: from lonpmailmx01.citrite.net ([10.30.203.162]) by LONPIPO01.EU.CITRIX.COM with ESMTP/TLS/RC4-MD5; 26 Mar 2013 10:10:17 +0000 Received: from [192.168.1.30] (10.30.249.104) by LONPMAILMX01.citrite.net (10.30.203.162) with Microsoft SMTP Server id 8.3.298.1; Tue, 26 Mar 2013 10:10:16 +0000 Message-ID: <51517407.8040406@citrix.com> Date: Tue, 26 Mar 2013 11:10:15 +0100 From: =?ISO-8859-1?Q?Roger_Pau_Monn=E9?= User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:17.0) Gecko/20130307 Thunderbird/17.0.4 MIME-Version: 1.0 To: Colin Percival Subject: Re: [Xen-users] Trouble booting FreeBSD i386 PV DomU References: <51516B06.8080704@citrix.com> <51516CAA.3040802@freebsd.org> In-Reply-To: <51516CAA.3040802@freebsd.org> Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 8bit Cc: xen-users , "freebsd-xen@freebsd.org" X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Mar 2013 10:10:19 -0000 On 26/03/13 10:38, Colin Percival wrote: > On 03/26/13 02:31, Roger Pau Monné wrote: >> Is Xen i386 PV broken? > > Not completely broken, but it's certainly not in a good state. I believe > it's broken with SMP, for example -- if the "crashed on cpu#7" in your > output means cpu#7 from the guest, it would certainly explain things. My guest only has one vcpu (vcpu#0): Domain 12 (vcpu#0) crashed on cpu#7 That is running on physical CPU 7 > HVM is the way to go with FreeBSD/Xen. Yes, I'm already working on that, and got vector callbacks working on both i386 and amd64 HVM guests, thanks to Justin T. Gibbs patch. Now I was trying to boot a PV guest to see how much breakage this change introduced to PV, but I'm not able to make it work, even without my patches. I've replied to this xen-users thread because the author seem to have a working FreeBSD DomU PV guest, and I was wondering how he did it. From my POV it seems like PV guests hasn't been working for a long time, since Xen 3.3 dropped support for non-PAE guests, and the FreeBSD kernel is detected as non-PAE. From owner-freebsd-xen@FreeBSD.ORG Tue Mar 26 10:16:17 2013 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 0D4D4FA1 for ; Tue, 26 Mar 2013 10:16:17 +0000 (UTC) (envelope-from bounces+73574-0602-freebsd-xen=freebsd.org@sendgrid.me) Received: from o3.shared.sendgrid.net (o3.shared.sendgrid.net [208.117.48.85]) by mx1.freebsd.org (Postfix) with SMTP id 96CF4F94 for ; Tue, 26 Mar 2013 10:16:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sendgrid.info; h=from :mime-version:to:cc:subject:references:in-reply-to:content-type :content-transfer-encoding; s=smtpapi; bh=KYzoUyc8myQsjybGG36duD guayw=; b=yxTyOOtpFxYTOOSiyA1TN0M+H86D+7SxQLCxjtTezOy1gSjmDGIa/V Hb5TjSWk4j9xi7UKE4VT0n31e2R2ZWyzVFGN3tATE7KQ8GFUtBFGkqSsZ2Mgdbod uDR1vQX5VU9Td5uWZJwB8jWlLrvhZCcTAGnkJGwEnM5I+/SlPN1NI= Received: by 10.36.109.142 with SMTP id mf35.14783.5151756F3 Tue, 26 Mar 2013 10:16:15 +0000 (UTC) Received: from mail.tarsnap.com (unknown [10.60.208.17]) by mi15 (SG) with ESMTP id 5151756f.37d7.3751169 for ; Tue, 26 Mar 2013 05:16:15 -0500 (CST) Received: (qmail 68775 invoked from network); 26 Mar 2013 10:16:14 -0000 Received: from unknown (HELO clamshell.daemonology.net) (127.0.0.1) by ec2-107-20-205-189.compute-1.amazonaws.com with ESMTP; 26 Mar 2013 10:16:14 -0000 Received: (qmail 16918 invoked from network); 26 Mar 2013 10:14:52 -0000 Received: from unknown (HELO clamshell.daemonology.net) (127.0.0.1) by clamshell.daemonology.net with SMTP; 26 Mar 2013 10:14:52 -0000 Message-ID: <5151751C.5080501@freebsd.org> Date: Tue, 26 Mar 2013 03:14:52 -0700 From: Colin Percival User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130314 Thunderbird/17.0.4 MIME-Version: 1.0 To: =?ISO-8859-1?Q?Roger_Pau_Monn=E9?= Subject: Re: [Xen-users] Trouble booting FreeBSD i386 PV DomU References: <51516B06.8080704@citrix.com> <51516CAA.3040802@freebsd.org> <51517407.8040406@citrix.com> In-Reply-To: <51517407.8040406@citrix.com> X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=ISO-8859-1 content-transfer-encoding: quoted-printable X-SG-EID: RUbAm5H8PjswBj/QH+sYVehaJogg3iBnZcyVi1bw/IyeDfxWpI4sSB754kJ2dwD5NMwGhucwKN6Wtyiu93eBnuDaoXG1nPSr08orKX7VvBwRqiCo+Tooeukh5HSB8oCO Cc: xen-users , "freebsd-xen@freebsd.org" X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Mar 2013 10:16:17 -0000 On 03/26/13 03:10, Roger Pau Monn=E9 wrote:=0D > On 26/03/13 10:38, Colin Percival wrote:=0D >> On 03/26/13 02:31, Roger Pau Monn=E9 wrote:=0D >>> Is Xen i386 PV broken?=0D >>=0D >> Not completely broken, but it's certainly not in a good state. I believ= e=0D >> it's broken with SMP, for example -- if the "crashed on cpu#7" in your= =0D >> output means cpu#7 from the guest, it would certainly explain things.=0D > =0D > My guest only has one vcpu (vcpu#0):=0D =0D Ok, I wasn't sure how to parse that output.=0D =0D >> HVM is the way to go with FreeBSD/Xen.=0D > =0D > Yes, I'm already working on that, and got vector callbacks working on=0D > both i386 and amd64 HVM guests, thanks to Justin T. Gibbs patch. Now I=0D > was trying to boot a PV guest to see how much breakage this change=0D > introduced to PV, but I'm not able to make it work, even without my patch= es.=0D > =0D > I've replied to this xen-users thread because the author seem to have a= =0D > working FreeBSD DomU PV guest, and I was wondering how he did it. From=0D > my POV it seems like PV guests hasn't been working for a long time,=0D > since Xen 3.3 dropped support for non-PAE guests, and the FreeBSD kernel= =0D > is detected as non-PAE.=0D =0D I had FreeBSD 8.2-RELEASE and a 9.0-CURRENT @ January 2011 running with PV= =0D in EC2 (http://www.daemonology.net/freebsd-on-ec2/, look for "t1.micro=0D instances only") and that used PAE. But it's entirely likely that somethin= g=0D got broken in the past two years and nobody noticed because nobody ever use= s=0D PV...=0D =0D -- =0D Colin Percival=0D Security Officer Emeritus, FreeBSD | The power to serve=0D Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid= =0D From owner-freebsd-xen@FreeBSD.ORG Tue Mar 26 15:29:12 2013 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 1545AA1C; Tue, 26 Mar 2013 15:29:12 +0000 (UTC) (envelope-from roger.pau@citrix.com) Received: from SMTP.EU.CITRIX.COM (smtp.eu.citrix.com [46.33.159.39]) by mx1.freebsd.org (Postfix) with ESMTP id 37A573EC; Tue, 26 Mar 2013 15:29:10 +0000 (UTC) X-IronPort-AV: E=Sophos;i="4.84,913,1355097600"; d="scan'208,223";a="2925604" Received: from lonpmailmx01.citrite.net ([10.30.203.162]) by LONPIPO01.EU.CITRIX.COM with ESMTP/TLS/RC4-MD5; 26 Mar 2013 15:29:10 +0000 Received: from [192.168.1.30] (10.30.249.104) by LONPMAILMX01.citrite.net (10.30.203.162) with Microsoft SMTP Server id 8.3.298.1; Tue, 26 Mar 2013 15:29:08 +0000 Message-ID: <5151BEC3.3030809@citrix.com> Date: Tue, 26 Mar 2013 16:29:07 +0100 From: =?ISO-8859-1?Q?Roger_Pau_Monn=E9?= User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:17.0) Gecko/20130307 Thunderbird/17.0.4 MIME-Version: 1.0 To: Colin Percival Subject: Re: [Xen-users] Trouble booting FreeBSD i386 PV DomU References: <51516B06.8080704@citrix.com> <51516CAA.3040802@freebsd.org> <51517407.8040406@citrix.com> <5151751C.5080501@freebsd.org> In-Reply-To: <5151751C.5080501@freebsd.org> Content-Type: multipart/mixed; boundary="------------050003050001070201070206" Cc: xen-users , "freebsd-xen@freebsd.org" X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Mar 2013 15:29:12 -0000 --------------050003050001070201070206 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 8bit On 26/03/13 11:14, Colin Percival wrote: > On 03/26/13 03:10, Roger Pau Monné wrote: >> On 26/03/13 10:38, Colin Percival wrote: >>> On 03/26/13 02:31, Roger Pau Monné wrote: >>>> Is Xen i386 PV broken? >>> >>> Not completely broken, but it's certainly not in a good state. I believe >>> it's broken with SMP, for example -- if the "crashed on cpu#7" in your >>> output means cpu#7 from the guest, it would certainly explain things. >> >> My guest only has one vcpu (vcpu#0): > > Ok, I wasn't sure how to parse that output. > >>> HVM is the way to go with FreeBSD/Xen. >> >> Yes, I'm already working on that, and got vector callbacks working on >> both i386 and amd64 HVM guests, thanks to Justin T. Gibbs patch. Now I >> was trying to boot a PV guest to see how much breakage this change >> introduced to PV, but I'm not able to make it work, even without my patches. >> >> I've replied to this xen-users thread because the author seem to have a >> working FreeBSD DomU PV guest, and I was wondering how he did it. From >> my POV it seems like PV guests hasn't been working for a long time, >> since Xen 3.3 dropped support for non-PAE guests, and the FreeBSD kernel >> is detected as non-PAE. > > I had FreeBSD 8.2-RELEASE and a 9.0-CURRENT @ January 2011 running with PV > in EC2 (http://www.daemonology.net/freebsd-on-ec2/, look for "t1.micro > instances only") and that used PAE. But it's entirely likely that something > got broken in the past two years and nobody noticed because nobody ever uses > PV... I've checked and 9.1 is working OK (at least boots and seems to be functional). I've also found the reason why HEAD doesn't work, and it's probably caused by the switch to clang, which seems to have problems with the ELFNOTE macro, here is a patch that makes i386 PV boot again: --- diff --git a/sys/i386/include/asmacros.h b/sys/i386/include/asmacros.h index c1c3f64..474dfe1 100644 --- a/sys/i386/include/asmacros.h +++ b/sys/i386/include/asmacros.h @@ -211,7 +211,7 @@ #ifdef __STDC__ #define ELFNOTE(name, type, desctype, descdata...) \ -.pushsection .note.name ; \ +.pushsection .note.name,"",@note ; \ .align 4 ; \ .long 2f - 1f /* namesz */ ; \ .long 4f - 3f /* descsz */ ; \ @@ -223,7 +223,7 @@ .popsection #else /* !__STDC__, i.e. -traditional */ #define ELFNOTE(name, type, desctype, descdata) \ -.pushsection .note.name ; \ +.pushsection .note.name,"",@note ; \ .align 4 ; \ .long 2f - 1f /* namesz */ ; \ .long 4f - 3f /* descsz */ ; \ --- But there's even more problems... rtc0: [XEN] xen_rtc_gettime rtc0: [XEN] xen_rtc_gettime: wallclock 1364233979 sec; 999999989 nsec rtc0: [XEN] xen_rtc_gettime: uptime 76890 sec; 166126554 nsec rtc0: [XEN] xen_rtc_gettime: TOD 1364310870 sec; 166126543 nsec start_init: trying /sbin/init pid 26 (sh), uid 0: exited on signal 4 panic: removing pages from non-current pmap cpuid = 0 KDB: enter: panic [ thread pid 26 tid 100037 ] Stopped at kdb_enter+0x3d: movl $0,kdb_why db> bt Tracing pid 26 tid 100037 td 0xc25e4900 kdb_enter(c03f8ad8,c03f8ad8,c04321de,ccf7699c,c04321de,...) at kdb_enter+0x3d/frame 0xccf76930 kassert_panic(c047c478,100,c04321de,ccf7699c,ccf7699c,...) at kassert_panic+0x232/frame 0xccf7696c kassert_panic(c04321de,c0431504,e0a,7bc,c044fe94,...) at kassert_panic+0xea/frame 0xccf76990 pmap_remove_pages(c2427638,c064bd74,8,c2427638,c248c8b8,...) at pmap_remove_pages+0xb3/frame 0xccf76a00 vmspace_exit(c25e4900,0,c03f2588,140,4,...) at vmspace_exit+0xb1/frame 0xccf76a28 exit1(c25e4900,4,1a,c248cacc,0,...) at exit1+0x654/frame 0xccf76a88 sigexit(c25e4900,4,c03f8b70,b0a,c0153a4f,...) at sigexit+0xc7f/frame 0xccf76c20 postsig(4,0,c03fee92,10d,0,...) at postsig+0x3b2/frame 0xccf76cdc ast(ccf76d18) at ast+0x388/frame 0xccf76d0c vm86_biosret() at vm86_biosret+0x9c/frame 0xbf7fcba8 --------------050003050001070201070206 Content-Type: text/plain; charset="UTF-8"; x-mac-type=0; x-mac-creator=0; name="0001-xen-fix-ELFNOTE-macro.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="0001-xen-fix-ELFNOTE-macro.patch" >From ef73078b37f283401159119538b8ac85bbd512d6 Mon Sep 17 00:00:00 2001 From: Roger Pau Monne Date: Tue, 26 Mar 2013 12:39:58 +0100 Subject: [PATCH] xen: fix ELFNOTE macro Add an empty flag and the corresponding type of section to pushsection. --- sys/i386/include/asmacros.h | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/sys/i386/include/asmacros.h b/sys/i386/include/asmacros.h index c1c3f64..474dfe1 100644 --- a/sys/i386/include/asmacros.h +++ b/sys/i386/include/asmacros.h @@ -211,7 +211,7 @@ #ifdef __STDC__ #define ELFNOTE(name, type, desctype, descdata...) \ -.pushsection .note.name ; \ +.pushsection .note.name,"",@note ; \ .align 4 ; \ .long 2f - 1f /* namesz */ ; \ .long 4f - 3f /* descsz */ ; \ @@ -223,7 +223,7 @@ .popsection #else /* !__STDC__, i.e. -traditional */ #define ELFNOTE(name, type, desctype, descdata) \ -.pushsection .note.name ; \ +.pushsection .note.name,"",@note ; \ .align 4 ; \ .long 2f - 1f /* namesz */ ; \ .long 4f - 3f /* descsz */ ; \ -- 1.7.7.5 (Apple Git-26) --------------050003050001070201070206--