From owner-freebsd-virtualization@FreeBSD.ORG Mon Jun 24 12:46:34 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id E2227550 for ; Mon, 24 Jun 2013 12:46:34 +0000 (UTC) (envelope-from mack@macktronics.com) Received: from borg.macktronics.com (borg.macktronics.com [209.181.253.68]) by mx1.freebsd.org (Postfix) with ESMTP id A2C0A1615 for ; Mon, 24 Jun 2013 12:46:34 +0000 (UTC) Received: from [172.22.30.42] (dulse.macktronics.com [209.181.253.69]) by borg.macktronics.com (Postfix) with ESMTP id 711A8266; Sun, 23 Jun 2013 09:02:04 -0500 (CDT) From: Dan Mack Date: Sun, 23 Jun 2013 09:02:04 -0500 Subject: bhyve hangs on init on recent 10-CURRENT To: freebsd-virtualization@freebsd.org Message-Id: <3F80B754-2000-4C30-8386-22E16FE415F4@macktronics.com> Mime-Version: 1.0 (Mac OS X Mail 6.5 \(1508\)) X-Mailer: Apple Mail (2.1508) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jun 2013 12:46:34 -0000 Is this just me? When I start a new bhyve instance, it hangs on = /sbin/init. top shows bhyve consuming about 1 host core of CPU. This = is on a real recent current. =20 root@darkstor:/vms/porter # ./vmrun.sh porter Launching virtual machine "porter" ... Consoles: userboot =20 FreeBSD/amd64 User boot, Revision 1.1 (root@darkstor.macktronics.com, Sat Jun 22 22:31:58 CDT 2013) Loading /boot/defaults/loader.conf=20 /boot/kernel/kernel text=3D0x6c21c7 data=3D0xa43b8+0xfa338 = syms=3D[0x8+0xe76a8+0x8+0x147ee8] \ ______ ____ _____ _____ =20 | ____| | _ \ / ____| __ \=20 | |___ _ __ ___ ___ | |_) | (___ | | | | | ___| '__/ _ \/ _ \| _ < \___ \| | | | | | | | | __/ __/| |_) |____) | |__| | | | | | | | || | | | |_| |_| \___|\___||____/|_____/|_____/ ``` = ` s` `.....---.......--.``` = -/ +------------Welcome to FreeBSD-----------+ +o .--` /y:` = +. | | yo`:. :o = `+- | 1. Back to Main Menu [Backspace] | y/ -/` = -o/ | 2. Load System [D]efaults | .- = ::/sy+:. | | / `-- = / | Boot Options: | `: = :` | 3. Safe [M]ode... off | `: = :` | 4. [S]ingle User. off | / = / | 5. [V]erbose..... On | .- = -. | | -- = -. | | `:` `:` | | .-- `--. | | .---.....----. +-----------------------------------------+ =20 Booting... GDB: no debug ports present KDB: debugger backends: ddb KDB: current backend: ddb SMAP type=3D01 base=3D0000000000000000 len=3D00000000000a0000 SMAP type=3D01 base=3D0000000000100000 len=3D0000000089300000 Table 'APIC' at 0xf0500 APIC: Found table at 0xf0500 APIC: Using the MADT enumerator. MADT: Found CPU APIC ID 0 ACPI ID 0: enabled SMP: Added CPU 0 (AP) MADT: Found CPU APIC ID 1 ACPI ID 1: enabled SMP: Added CPU 1 (AP) Copyright (c) 1992-2013 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights = reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.0-CURRENT #9 r251956: Tue Jun 18 12:25:25 CDT 2013 root@darkstor.macktronics.com:/usr/obj/usr/src/sys/MACKGEN amd64 FreeBSD clang version 3.3 (tags/RELEASE_33/final 183502) 20130610 Preloaded elf kernel "/boot/kernel/kernel" at 0xffffffff80e91000. Hypervisor: Origin =3D "BHyVBHyVBHyV" Calibrating TSC clock ... TSC clock: 3299774008 Hz CPU: Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz (3299.77-MHz K8-class = CPU) Origin =3D "GenuineIntel" Id =3D 0x306a9 Family =3D 0x6 Model =3D = 0x3a Stepping =3D 9 = Features=3D0x8fa3ab7f = Features2=3D0xe3bae257 AMD Features=3D0x28100800 AMD Features2=3D0x1 TSC: P-state invariant real memory =3D 2302672896 (2196 MB) Physical memory chunk(s): 0x0000000000001000 - 0x000000000009bfff, 634880 bytes (155 pages) 0x0000000000100000 - 0x00000000001fffff, 1048576 bytes (256 pages) 0x0000000000eb1000 - 0x0000000085bc1fff, 2228293632 bytes (544017 pages) avail memory =3D 2133233664 (2034 MB) Event timer "LAPIC" quality 400 ACPI APIC Table: INTR: Adding local APIC 1 as a target FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs FreeBSD/SMP: 2 package(s) x 1 core(s) cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 x86bios: IVT 0x000000-0x0004ff at 0xfffffe0000000000 x86bios: SSEG 0x001000-0x001fff at 0xffffff800022e000 x86bios: ROM 0x0a0000-0x0fefff at 0xfffffe00000a0000 APIC: CPU 0 has ACPI ID 0 APIC: CPU 1 has ACPI ID 1 random device not loaded; using insecure entropy ULE: setup cpu 0 ULE: setup cpu 1 ACPI: RSDP 0xf0400 00024 (v02 BHYVE ) ACPI: XSDT 0xf0480 00034 (v01 BHYVE BVXSDT 00000001 INTL 20130517) ACPI: APIC 0xf0500 00052 (v01 BHYVE BVMADT 00000001 INTL 20130517) ACPI: FACP 0xf0600 0010C (v05 BHYVE BVFACP 00000001 INTL 20130517) ACPI: DSDT 0xf0800 000F2 (v02 BHYVE BVDSDT 00000001 INTL 20130517) ACPI: FACS 0xf0780 00040 MADT: Found IO APIC ID 2, Interrupt 0 at 0xfec00000 ioapic0: Changing APIC ID to 2 ioapic0: Routing external 8259A's -> intpin 0 MADT: Interrupt override: source 9, irq 9 ioapic0: intpin 9 trigger: level ioapic0: intpin 9 polarity: low ioapic0 irqs 0-16 on motherboard cpu0 BSP: ID: 0x00000000 VER: 0x00000011 LDR: 0x00000000 DFR: 0xffffffff lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 null: random: module_register_init: MOD_LOAD (vesa, 0xffffffff807b7950, 0) error 19 io: kbd0 at kbdmux0 nfslock: pseudo-device mem: acpi0: on motherboard ACPI: All ACPI Tables successfully acquired ioapic0: routing intpin 9 (ISA IRQ 9) to lapic 0 vector 48 atrtc0: port 0x70-0x71,0x72-0x77 irq 8 on acpi0 atrtc0: registered as a time-of-day clock (resolution 1000000us, = adjustment 0.500000000s) ioapic0: routing intpin 8 (ISA IRQ 8) to lapic 0 vector 49 Event timer "RTC" frequency 32768 Hz quality 0 ACPI timer: 1/6 1/5 1/6 1/5 1/7 1/5 1/4 1/4 1/4 1/4 -> 10 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <32-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pcib0: decoding 4 range 0-0xcf7 pcib0: decoding 4 range 0xd00-0xffff pcib0: could not get PCI interrupt routing table for \_SB_.PCI0 - = AE_NOT_FOUND pci0: on pcib0 pci0: domain=3D0, physical bus=3D0 found-> vendor=3D0x1275, dev=3D0x1275, revid=3D0x00 domain=3D0, bus=3D0, slot=3D0, func=3D0 class=3D06-00-00, hdrtype=3D0x01, mfdev=3D0 cmdreg=3D0x0007, statreg=3D0x0010, cachelnsz=3D0 (dwords) lattimer=3D0x00 (0 ns), mingnt=3D0x00 (0 ns), maxlat=3D0x00 (0 = ns) found-> vendor=3D0x1af4, dev=3D0x1000, revid=3D0x00 domain=3D0, bus=3D0, slot=3D1, func=3D0 class=3D02-00-00, hdrtype=3D0x00, mfdev=3D0 cmdreg=3D0x0007, statreg=3D0x0010, cachelnsz=3D0 (dwords) lattimer=3D0x00 (0 ns), mingnt=3D0x00 (0 ns), maxlat=3D0x00 (0 = ns) MSI-X supports 3 messages in map 0x14 map[10]: type I/O Port, range 32, base 0x2000, size 5, enabled pcib0: allocated type 4 (0x2000-0x201f) for rid 10 of pci0:0:1:0 map[14]: type Memory, range 32, base 0xc0000000, size 13, = enabled found-> vendor=3D0x1af4, dev=3D0x1001, revid=3D0x00 domain=3D0, bus=3D0, slot=3D2, func=3D0 class=3D01-00-00, hdrtype=3D0x00, mfdev=3D0 cmdreg=3D0x0007, statreg=3D0x0010, cachelnsz=3D0 (dwords) lattimer=3D0x00 (0 ns), mingnt=3D0x00 (0 ns), maxlat=3D0x00 (0 = ns) MSI-X supports 2 messages in map 0x14 map[10]: type I/O Port, range 32, base 0x2040, size 6, enabled pcib0: allocated type 4 (0x2040-0x207f) for rid 10 of pci0:0:2:0 map[14]: type Memory, range 32, base 0xc0002000, size 13, = enabled found-> vendor=3D0x1af4, dev=3D0x1001, revid=3D0x00 domain=3D0, bus=3D0, slot=3D3, func=3D0 class=3D01-00-00, hdrtype=3D0x00, mfdev=3D0 cmdreg=3D0x0007, statreg=3D0x0010, cachelnsz=3D0 (dwords) lattimer=3D0x00 (0 ns), mingnt=3D0x00 (0 ns), maxlat=3D0x00 (0 = ns) MSI-X supports 2 messages in map 0x14 map[10]: type I/O Port, range 32, base 0x2080, size 6, enabled pcib0: allocated type 4 (0x2080-0x20bf) for rid 10 of pci0:0:3:0 map[14]: type Memory, range 32, base 0xc0004000, size 13, = enabled found-> vendor=3D0x131f, dev=3D0x2000, revid=3D0x00 domain=3D0, bus=3D0, slot=3D31, func=3D0 class=3D07-00-00, hdrtype=3D0x00, mfdev=3D0 cmdreg=3D0x0007, statreg=3D0x0000, cachelnsz=3D0 (dwords) lattimer=3D0x00 (0 ns), mingnt=3D0x00 (0 ns), maxlat=3D0x00 (0 = ns) intpin=3Da, irq=3D4 map[10]: type I/O Port, range 32, base 0x3f8, size 3, enabled pcib0: allocated type 4 (0x3f8-0x3ff) for rid 10 of pci0:0:31:0 pcib0: no PRT entry for 0.31.INTA virtio_pci0: port 0x2000-0x201f mem = 0xc0000000-0xc0001fff at device 1.0 on pci0 vtnet0: on virtio_pci0 virtio_pci0: host features: 0x1018020 = virtio_pci0: negotiated features: 0x1018020 = virtio_pci0: virtqueue 1 (vtnet0 transmit) requested indirect = descriptors but not negotiated vtnet0: bpf attached vtnet0: Ethernet address: 00:a0:98:fd:e2:7e virtio_pci0: attempting to allocate 3 MSI-X vectors (3 supported) msi: routing MSI-X IRQ 256 to local APIC 0 vector 50 msi: routing MSI-X IRQ 257 to local APIC 0 vector 51 msi: routing MSI-X IRQ 258 to local APIC 0 vector 52 virtio_pci0: using IRQs 256-258 for MSI-X virtio_pci0: using per VQ MSIX interrupts virtio_pci1: port 0x2040-0x207f mem = 0xc0002000-0xc0003fff at device 2.0 on pci0 vtblk0: on virtio_pci1 virtio_pci1: host features: 0x10000004 virtio_pci1: negotiated features: 0x10000004 virtio_pci1: attempting to allocate 2 MSI-X vectors (2 supported) msi: routing MSI-X IRQ 259 to local APIC 0 vector 53 msi: routing MSI-X IRQ 260 to local APIC 0 vector 54 virtio_pci1: using IRQs 259-260 for MSI-X virtio_pci1: using per VQ MSIX interrupts vtblk0: 18432MB (37748736 512 byte sectors) virtio_pci2: port 0x2080-0x20bf mem = 0xc0004000-0xc0005fff at device 3.0 on pci0 vtblk1: on virtio_pci2 virtio_pci2: host features: 0x10000004 virtio_pci2: negotiated features: 0x10000004 virtio_pci2: attempting to allocate 2 MSI-X vectors (2 supported) msi: routing MSI-X IRQ 261 to local APIC 0 vector 55 msi: routing MSI-X IRQ 262 to local APIC 0 vector 56 virtio_pci2: using IRQs 261-262 for MSI-X virtio_pci2: using per VQ MSIX interrupts vtblk1: 683MB (1400648 512 byte sectors) uart2: <16550 or compatible> port 0x3f8-0x3ff irq 4 at device 31.0 on = pci0 ioapic0: routing intpin 4 (ISA IRQ 4) to lapic 0 vector 57 uart2: fast interrupt uart2: console (9600,n,8,1) acpi0: wakeup code va 0xffffff80002c5000 pa 0x4000 ctl: CAM Target Layer loaded Device configuration finished. procfs registered lapic: Divisor 2, Frequency 1650046000 Hz Timecounters tick every 10.000 msec vlan: initialized, using hash tables with chaining tcp_init: net.inet.tcp.tcbhashsize auto tuned to 32768 lo0: bpf attached GEOM: new disk vtbd0 GEOM: new disk vtbd1 SMP: AP CPU #1 Launched! cpu1 AP: ID: 0x01000000 VER: 0x00000011 LDR: 0x00000000 DFR: 0xffffffff lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 ioapic0: routing intpin 9 (ISA IRQ 9) to lapic 1 vector 48 msi: Assigning MSI-X IRQ 257 to local APIC 1 vector 49 msi: Assigning MSI-X IRQ 259 to local APIC 1 vector 50 msi: Assigning MSI-X IRQ 261 to local APIC 1 vector 51 TSC timecounter discards lower 1 bit(s) Timecounter "TSC-low" frequency 1649887004 Hz quality -100 Trying to mount root from cd9660:/dev/iso9660/FREEBSD_INSTALL [ro]... cd9660: RockRidge Extension start_init: trying /sbin/init [ hangs indefinitely ] CPUs on the host look like this: root@darkstor:~ # top -bHI last pid: 7781; load averages: 1.00, 0.93, 0.62 up 0+00:51:59 = 07:32:45 28 processes: 2 running, 26 sleeping Mem: 31M Active, 27M Inact, 5408M Wired, 4096K Cache, 26G Free ARC: 1642M Total, 628M MFU, 992M MRU, 16K Anon, 7038K Header, 15M Other Swap:=20 PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND 7757 root 88 0 2221M 5160K CPU1 1 2:30 54.20% = bhyve{porter vcpu 0} 7757 root 36 0 2221M 5160K vmidle 3 5:02 48.19% = bhyve{porter vcpu 1} root@darkstor:~ # uname -a FreeBSD darkstor.macktronics.com 10.0-CURRENT FreeBSD 10.0-CURRENT #11 = r252104: Sat Jun 22 22:45:53 CDT 2013 = root@darkstor.macktronics.com:/usr/obj/usr/src/sys/MACKGEN amd64