From owner-freebsd-virtualization@FreeBSD.ORG Tue Jun 25 14:07:45 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 7535CD93 for ; Tue, 25 Jun 2013 14:07:45 +0000 (UTC) (envelope-from mack@macktronics.com) Received: from borg.macktronics.com (borg.macktronics.com [209.181.253.68]) by mx1.freebsd.org (Postfix) with ESMTP id 2A63B1D11 for ; Tue, 25 Jun 2013 14:07:44 +0000 (UTC) Received: from olive.macktronics.com (olive.macktronics.com [209.181.253.67]) by borg.macktronics.com (Postfix) with ESMTP id 249496B8; Tue, 25 Jun 2013 09:07:38 -0500 (CDT) Date: Tue, 25 Jun 2013 09:07:38 -0500 (CDT) From: Dan Mack To: Neel Natu Subject: Re: bhyve hangs on init on recent 10-CURRENT In-Reply-To: Message-ID: References: <3F80B754-2000-4C30-8386-22E16FE415F4@macktronics.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jun 2013 14:07:45 -0000 Thanks, I think I just tracked it down to the release.iso image I built with with r251956. I tried an older release.iso and it didn't hang. I'll double check with a recent build and see if this continues or not, I suspect it's just a problem with that build. Dan On Mon, 24 Jun 2013, Neel Natu wrote: > Hi Dan, > > On Sun, Jun 23, 2013 at 7:02 AM, Dan Mack wrote: >> Is this just me? When I start a new bhyve instance, it hangs on /sbin/init. top shows bhyve consuming about 1 host core of CPU. This is on a real recent current. >> > > I don't see it on my system with this: > FreeBSD 10.0-CURRENT (GENERIC) #0: Sun Jun 23 02:55:37 UTC 2013 > > My system has a much lower clock speed that yours - 2GHz versus > 3.2GHz, but other than that I kept everything else similar to your > setup. > > Is this a new system or were you able to use this successfully in the > past to run bhyve? > > best > Neel > >> >> >> root@darkstor:/vms/porter # ./vmrun.sh porter >> Launching virtual machine "porter" ... >> Consoles: userboot >> >> FreeBSD/amd64 User boot, Revision 1.1 >> (root@darkstor.macktronics.com, Sat Jun 22 22:31:58 CDT 2013) >> Loading /boot/defaults/loader.conf >> /boot/kernel/kernel text=0x6c21c7 data=0xa43b8+0xfa338 syms=[0x8+0xe76a8+0x8+0x147ee8] >> \ >> ______ ____ _____ _____ >> | ____| | _ \ / ____| __ \ >> | |___ _ __ ___ ___ | |_) | (___ | | | | >> | ___| '__/ _ \/ _ \| _ < \___ \| | | | >> | | | | | __/ __/| |_) |____) | |__| | >> | | | | | | || | | | >> |_| |_| \___|\___||____/|_____/|_____/ ``` ` >> s` `.....---.......--.``` -/ >> +------------Welcome to FreeBSD-----------+ +o .--` /y:` +. >> | | yo`:. :o `+- >> | 1. Back to Main Menu [Backspace] | y/ -/` -o/ >> | 2. Load System [D]efaults | .- ::/sy+:. >> | | / `-- / >> | Boot Options: | `: :` >> | 3. Safe [M]ode... off | `: :` >> | 4. [S]ingle User. off | / / >> | 5. [V]erbose..... On | .- -. >> | | -- -. >> | | `:` `:` >> | | .-- `--. >> | | .---.....----. >> +-----------------------------------------+ >> >> >> Booting... >> GDB: no debug ports present >> KDB: debugger backends: ddb >> KDB: current backend: ddb >> SMAP type=01 base=0000000000000000 len=00000000000a0000 >> SMAP type=01 base=0000000000100000 len=0000000089300000 >> Table 'APIC' at 0xf0500 >> APIC: Found table at 0xf0500 >> APIC: Using the MADT enumerator. >> MADT: Found CPU APIC ID 0 ACPI ID 0: enabled >> SMP: Added CPU 0 (AP) >> MADT: Found CPU APIC ID 1 ACPI ID 1: enabled >> SMP: Added CPU 1 (AP) >> Copyright (c) 1992-2013 The FreeBSD Project. >> Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 >> The Regents of the University of California. All rights reserved. >> FreeBSD is a registered trademark of The FreeBSD Foundation. >> FreeBSD 10.0-CURRENT #9 r251956: Tue Jun 18 12:25:25 CDT 2013 >> root@darkstor.macktronics.com:/usr/obj/usr/src/sys/MACKGEN amd64 >> FreeBSD clang version 3.3 (tags/RELEASE_33/final 183502) 20130610 >> Preloaded elf kernel "/boot/kernel/kernel" at 0xffffffff80e91000. >> Hypervisor: Origin = "BHyVBHyVBHyV" >> Calibrating TSC clock ... TSC clock: 3299774008 Hz >> CPU: Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz (3299.77-MHz K8-class CPU) >> Origin = "GenuineIntel" Id = 0x306a9 Family = 0x6 Model = 0x3a Stepping = 9 >> Features=0x8fa3ab7f >> Features2=0xe3bae257 >> AMD Features=0x28100800 >> AMD Features2=0x1 >> TSC: P-state invariant >> real memory = 2302672896 (2196 MB) >> Physical memory chunk(s): >> 0x0000000000001000 - 0x000000000009bfff, 634880 bytes (155 pages) >> 0x0000000000100000 - 0x00000000001fffff, 1048576 bytes (256 pages) >> 0x0000000000eb1000 - 0x0000000085bc1fff, 2228293632 bytes (544017 pages) >> avail memory = 2133233664 (2034 MB) >> Event timer "LAPIC" quality 400 >> ACPI APIC Table: >> INTR: Adding local APIC 1 as a target >> FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs >> FreeBSD/SMP: 2 package(s) x 1 core(s) >> cpu0 (BSP): APIC ID: 0 >> cpu1 (AP): APIC ID: 1 >> x86bios: IVT 0x000000-0x0004ff at 0xfffffe0000000000 >> x86bios: SSEG 0x001000-0x001fff at 0xffffff800022e000 >> x86bios: ROM 0x0a0000-0x0fefff at 0xfffffe00000a0000 >> APIC: CPU 0 has ACPI ID 0 >> APIC: CPU 1 has ACPI ID 1 >> random device not loaded; using insecure entropy >> ULE: setup cpu 0 >> ULE: setup cpu 1 >> ACPI: RSDP 0xf0400 00024 (v02 BHYVE ) >> ACPI: XSDT 0xf0480 00034 (v01 BHYVE BVXSDT 00000001 INTL 20130517) >> ACPI: APIC 0xf0500 00052 (v01 BHYVE BVMADT 00000001 INTL 20130517) >> ACPI: FACP 0xf0600 0010C (v05 BHYVE BVFACP 00000001 INTL 20130517) >> ACPI: DSDT 0xf0800 000F2 (v02 BHYVE BVDSDT 00000001 INTL 20130517) >> ACPI: FACS 0xf0780 00040 >> MADT: Found IO APIC ID 2, Interrupt 0 at 0xfec00000 >> ioapic0: Changing APIC ID to 2 >> ioapic0: Routing external 8259A's -> intpin 0 >> MADT: Interrupt override: source 9, irq 9 >> ioapic0: intpin 9 trigger: level >> ioapic0: intpin 9 polarity: low >> ioapic0 irqs 0-16 on motherboard >> cpu0 BSP: >> ID: 0x00000000 VER: 0x00000011 LDR: 0x00000000 DFR: 0xffffffff >> lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff >> timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 >> null: >> random: >> module_register_init: MOD_LOAD (vesa, 0xffffffff807b7950, 0) error 19 >> io: >> kbd0 at kbdmux0 >> nfslock: pseudo-device >> mem: >> acpi0: on motherboard >> ACPI: All ACPI Tables successfully acquired >> ioapic0: routing intpin 9 (ISA IRQ 9) to lapic 0 vector 48 >> atrtc0: port 0x70-0x71,0x72-0x77 irq 8 on acpi0 >> atrtc0: registered as a time-of-day clock (resolution 1000000us, adjustment 0.500000000s) >> ioapic0: routing intpin 8 (ISA IRQ 8) to lapic 0 vector 49 >> Event timer "RTC" frequency 32768 Hz quality 0 >> ACPI timer: 1/6 1/5 1/6 1/5 1/7 1/5 1/4 1/4 1/4 1/4 -> 10 >> Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 >> acpi_timer0: <32-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 >> pcib0: port 0xcf8-0xcff on acpi0 >> pcib0: decoding 4 range 0-0xcf7 >> pcib0: decoding 4 range 0xd00-0xffff >> pcib0: could not get PCI interrupt routing table for \_SB_.PCI0 - AE_NOT_FOUND >> pci0: on pcib0 >> pci0: domain=0, physical bus=0 >> found-> vendor=0x1275, dev=0x1275, revid=0x00 >> domain=0, bus=0, slot=0, func=0 >> class=06-00-00, hdrtype=0x01, mfdev=0 >> cmdreg=0x0007, statreg=0x0010, cachelnsz=0 (dwords) >> lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) >> found-> vendor=0x1af4, dev=0x1000, revid=0x00 >> domain=0, bus=0, slot=1, func=0 >> class=02-00-00, hdrtype=0x00, mfdev=0 >> cmdreg=0x0007, statreg=0x0010, cachelnsz=0 (dwords) >> lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) >> MSI-X supports 3 messages in map 0x14 >> map[10]: type I/O Port, range 32, base 0x2000, size 5, enabled >> pcib0: allocated type 4 (0x2000-0x201f) for rid 10 of pci0:0:1:0 >> map[14]: type Memory, range 32, base 0xc0000000, size 13, enabled >> found-> vendor=0x1af4, dev=0x1001, revid=0x00 >> domain=0, bus=0, slot=2, func=0 >> class=01-00-00, hdrtype=0x00, mfdev=0 >> cmdreg=0x0007, statreg=0x0010, cachelnsz=0 (dwords) >> lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) >> MSI-X supports 2 messages in map 0x14 >> map[10]: type I/O Port, range 32, base 0x2040, size 6, enabled >> pcib0: allocated type 4 (0x2040-0x207f) for rid 10 of pci0:0:2:0 >> map[14]: type Memory, range 32, base 0xc0002000, size 13, enabled >> found-> vendor=0x1af4, dev=0x1001, revid=0x00 >> domain=0, bus=0, slot=3, func=0 >> class=01-00-00, hdrtype=0x00, mfdev=0 >> cmdreg=0x0007, statreg=0x0010, cachelnsz=0 (dwords) >> lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) >> MSI-X supports 2 messages in map 0x14 >> map[10]: type I/O Port, range 32, base 0x2080, size 6, enabled >> pcib0: allocated type 4 (0x2080-0x20bf) for rid 10 of pci0:0:3:0 >> map[14]: type Memory, range 32, base 0xc0004000, size 13, enabled >> found-> vendor=0x131f, dev=0x2000, revid=0x00 >> domain=0, bus=0, slot=31, func=0 >> class=07-00-00, hdrtype=0x00, mfdev=0 >> cmdreg=0x0007, statreg=0x0000, cachelnsz=0 (dwords) >> lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) >> intpin=a, irq=4 >> map[10]: type I/O Port, range 32, base 0x3f8, size 3, enabled >> pcib0: allocated type 4 (0x3f8-0x3ff) for rid 10 of pci0:0:31:0 >> pcib0: no PRT entry for 0.31.INTA >> virtio_pci0: port 0x2000-0x201f mem 0xc0000000-0xc0001fff at device 1.0 on pci0 >> vtnet0: on virtio_pci0 >> virtio_pci0: host features: 0x1018020 >> virtio_pci0: negotiated features: 0x1018020 >> virtio_pci0: virtqueue 1 (vtnet0 transmit) requested indirect descriptors but not negotiated >> vtnet0: bpf attached >> vtnet0: Ethernet address: 00:a0:98:fd:e2:7e >> virtio_pci0: attempting to allocate 3 MSI-X vectors (3 supported) >> msi: routing MSI-X IRQ 256 to local APIC 0 vector 50 >> msi: routing MSI-X IRQ 257 to local APIC 0 vector 51 >> msi: routing MSI-X IRQ 258 to local APIC 0 vector 52 >> virtio_pci0: using IRQs 256-258 for MSI-X >> virtio_pci0: using per VQ MSIX interrupts >> virtio_pci1: port 0x2040-0x207f mem 0xc0002000-0xc0003fff at device 2.0 on pci0 >> vtblk0: on virtio_pci1 >> virtio_pci1: host features: 0x10000004 >> virtio_pci1: negotiated features: 0x10000004 >> virtio_pci1: attempting to allocate 2 MSI-X vectors (2 supported) >> msi: routing MSI-X IRQ 259 to local APIC 0 vector 53 >> msi: routing MSI-X IRQ 260 to local APIC 0 vector 54 >> virtio_pci1: using IRQs 259-260 for MSI-X >> virtio_pci1: using per VQ MSIX interrupts >> vtblk0: 18432MB (37748736 512 byte sectors) >> virtio_pci2: port 0x2080-0x20bf mem 0xc0004000-0xc0005fff at device 3.0 on pci0 >> vtblk1: on virtio_pci2 >> virtio_pci2: host features: 0x10000004 >> virtio_pci2: negotiated features: 0x10000004 >> virtio_pci2: attempting to allocate 2 MSI-X vectors (2 supported) >> msi: routing MSI-X IRQ 261 to local APIC 0 vector 55 >> msi: routing MSI-X IRQ 262 to local APIC 0 vector 56 >> virtio_pci2: using IRQs 261-262 for MSI-X >> virtio_pci2: using per VQ MSIX interrupts >> vtblk1: 683MB (1400648 512 byte sectors) >> uart2: <16550 or compatible> port 0x3f8-0x3ff irq 4 at device 31.0 on pci0 >> ioapic0: routing intpin 4 (ISA IRQ 4) to lapic 0 vector 57 >> uart2: fast interrupt >> uart2: console (9600,n,8,1) >> acpi0: wakeup code va 0xffffff80002c5000 pa 0x4000 >> ctl: CAM Target Layer loaded >> Device configuration finished. >> procfs registered >> lapic: Divisor 2, Frequency 1650046000 Hz >> Timecounters tick every 10.000 msec >> vlan: initialized, using hash tables with chaining >> tcp_init: net.inet.tcp.tcbhashsize auto tuned to 32768 >> lo0: bpf attached >> GEOM: new disk vtbd0 >> GEOM: new disk vtbd1 >> SMP: AP CPU #1 Launched! >> cpu1 AP: >> ID: 0x01000000 VER: 0x00000011 LDR: 0x00000000 DFR: 0xffffffff >> lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff >> timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 >> ioapic0: routing intpin 9 (ISA IRQ 9) to lapic 1 vector 48 >> msi: Assigning MSI-X IRQ 257 to local APIC 1 vector 49 >> msi: Assigning MSI-X IRQ 259 to local APIC 1 vector 50 >> msi: Assigning MSI-X IRQ 261 to local APIC 1 vector 51 >> TSC timecounter discards lower 1 bit(s) >> Timecounter "TSC-low" frequency 1649887004 Hz quality -100 >> Trying to mount root from cd9660:/dev/iso9660/FREEBSD_INSTALL [ro]... >> cd9660: RockRidge Extension >> start_init: trying /sbin/init >> >> [ hangs indefinitely ] >> >> CPUs on the host look like this: >> >> root@darkstor:~ # top -bHI >> last pid: 7781; load averages: 1.00, 0.93, 0.62 up 0+00:51:59 07:32:45 >> 28 processes: 2 running, 26 sleeping >> >> Mem: 31M Active, 27M Inact, 5408M Wired, 4096K Cache, 26G Free >> ARC: 1642M Total, 628M MFU, 992M MRU, 16K Anon, 7038K Header, 15M Other >> Swap: >> >> >> PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND >> 7757 root 88 0 2221M 5160K CPU1 1 2:30 54.20% bhyve{porter vcpu 0} >> 7757 root 36 0 2221M 5160K vmidle 3 5:02 48.19% bhyve{porter vcpu 1} >> >> root@darkstor:~ # uname -a >> FreeBSD darkstor.macktronics.com 10.0-CURRENT FreeBSD 10.0-CURRENT #11 r252104: Sat Jun 22 22:45:53 CDT 2013 root@darkstor.macktronics.com:/usr/obj/usr/src/sys/MACKGEN amd64 >> >> >> >> >> >> _______________________________________________ >> freebsd-virtualization@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization >> To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" > dan -- Dan Mack