From owner-freebsd-xen@freebsd.org Thu Oct 10 23:23:33 2019 Return-Path: Delivered-To: freebsd-xen@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 19B5D134CFC for ; Thu, 10 Oct 2019 23:23:33 +0000 (UTC) (envelope-from buhrow@lothlorien.nfbcal.org) Received: from lothlorien.nfbcal.org (ns.NFBCAL.ORG [157.22.230.125]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "nfbcal.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46q6b13RSQz3Clt for ; Thu, 10 Oct 2019 23:23:24 +0000 (UTC) (envelope-from buhrow@lothlorien.nfbcal.org) Received: from lothlorien.nfbcal.org (localhost [127.0.0.1]) by lothlorien.nfbcal.org (8.15.2/8.14.1-NFBNETBSD) with ESMTPS id x9ANNCMM016624 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 10 Oct 2019 16:23:15 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.97.3 at lothlorien.nfbcal.org Received: (from buhrow@localhost) by lothlorien.nfbcal.org (8.15.2/8.12.11) id x9ANNACe003999; Thu, 10 Oct 2019 16:23:10 -0700 (PDT) Message-Id: <201910102323.x9ANNACe003999@lothlorien.nfbcal.org> From: Brian Buhrow Date: Thu, 10 Oct 2019 16:23:10 -0700 X-Mailer: Mail User's Shell (7.2.6 beta(4.pl1)+dynamic 20000103) To: freebsd-xen@freebsd.org Subject: i/o is very slow on FreeBSD dom0 with Xen-4.12 and Freebsd-12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (lothlorien.nfbcal.org [127.0.0.1]); Thu, 10 Oct 2019 16:23:15 -0700 (PDT) X-Rspamd-Queue-Id: 46q6b13RSQz3Clt X-Spamd-Bar: ++++ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of buhrow@lothlorien.nfbcal.org designates 157.22.230.125 as permitted sender) smtp.mailfrom=buhrow@lothlorien.nfbcal.org X-Spamd-Result: default: False [4.01 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.34)[-0.335,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+a:ns.nfbcal.org:c]; R_BAD_CTE_7BIT(3.50)[7bit]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; DMARC_NA(0.00)[nfbcal.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_SPAM_LONG(0.85)[0.853,0]; IP_SCORE(-0.01)[country: US(-0.05)]; FORGED_SENDER(0.30)[buhrow@nfbcal.org,buhrow@lothlorien.nfbcal.org]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:7091, ipnet:157.22.0.0/16, country:US]; FROM_NEQ_ENVFROM(0.00)[buhrow@nfbcal.org,buhrow@lothlorien.nfbcal.org]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Oct 2019 23:23:33 -0000 hello. I'm trying to get a FreeBSD-12 xen server up and running and have run into the issue that i/o seems to be very slow for the dom0, which is FreeBSD-12.0-stable, and Xen is xen-4.12. I followed the instructions for making FreeBSD a dom0 as written in 21.8 of the FreeBSD handbook. All went well until I rebooted the machine with the /boot/xen kern loading. The dom0 boots, but takes about 2 hours to get to the login prompt and, on the console, when ever I press control-t, it says what ever process is currently running is in biord. I'm familiar with NetBSD and how to make it run xen, and am familiar with FreeBSD as well. Stil, I'm sure I'm doing something wrong that's simple, or, I missed some instruction that will seem obvious in retrospect. If anyone can help me shed light on the trouble, that would be great! Below are my /boot/loader.conf and a transcript of a sample boot. Note that it took about 2 hours to get from the initial loading of the kernel to the getty error messages. -thanks -Brian boot_serial="YES" # -h: Use serial console comconsole_speed="115200" # Set the current serial console speed comconsole_port="0x2f8" # Set the current serial console port #console="vidconsole" # A comma separated list of console(s) console="comconsole" # A comma separated list of console(s) geom_mirror_load="YES" # RAID1 disk driver (see gmirror(8)) ipmi_load="YES" # Ripmi driver if_tap_load="YES" # Load bridge driver. if_vlan_load="YES" # Load vlan driver. # Turn on Xen (BB 10/08/2019) xen_kernel="/boot/xen" xen_cmdline="dom0_mem=4096m dom0_max_vcpus=4 dom0=pvh console=com2,vga clocksource=hpet com2=115200,8n1 guest_loglvl=all loglvl=all" Loading kernel... Xen 4.12.1 (XEN) Xen version 4.12.1 (buhrow@) (FreeBSD clang version 6.0.1 (tags/RELEASE_601/final 335540) (based on LLVM 6.0.1)) debug=n Thu Oct 3 11:34:20 PDT 2019 (XEN) Latest ChangeSet: (XEN) Bootloader: FreeBSD Loader (XEN) Command line: dom0_mem=4096m dom0_max_vcpus=4 dom0=pvh console=com2,vga clocksource=hpet com2=115200,8n1 guest_loglvl=all loglvl=all (XEN) Xen image load base address: 0 (XEN) Video information: (XEN) VGA is text mode 80x25, font 8x16 (XEN) VBE/DDC methods: none; EDID transfer time: 0 seconds (XEN) EDID info not retrieved because no DDC retrieval method detected (XEN) Disc information: (XEN) Found 4 MBR signatures (XEN) Found 4 EDD information structures (XEN) Xen-e820 RAM map: (XEN) 0000000000000000 - 0000000000099800 (usable) (XEN) 0000000000099800 - 00000000000a0000 (reserved) (XEN) 00000000000e0000 - 0000000000100000 (reserved) (XEN) 0000000000100000 - 00000000bf780000 (usable) (XEN) 00000000bf78e000 - 00000000bf790000 type 9 (XEN) 00000000bf790000 - 00000000bf79e000 (ACPI data) (XEN) 00000000bf79e000 - 00000000bf7d0000 (ACPI NVS) (XEN) 00000000bf7d0000 - 00000000bf7e0000 (reserved) (XEN) 00000000bf7ec000 - 00000000c0000000 (reserved) (XEN) 00000000e0000000 - 00000000f0000000 (reserved) (XEN) 00000000fee00000 - 00000000fee01000 (reserved) (XEN) 00000000ffc00000 - 0000000100000000 (reserved) (XEN) 0000000100000000 - 0000001840000000 (usable) (XEN) New Xen image base address: 0xbf000000 (XEN) ACPI: RSDP 000FACE0, 0024 (r2 ACPIAM) (XEN) ACPI: XSDT BF790100, 008C (r1 SMCI 20120803 MSFT 97) (XEN) ACPI: FACP BF790290, 00F4 (r3 080312 FACP1521 20120803 MSFT 97) (XEN) ACPI: DSDT BF7906A0, 6580 (r1 10600 10600000 0 INTL 20051117) (XEN) ACPI: FACS BF79E000, 0040 (XEN) ACPI: APIC BF790390, 011E (r1 080312 APIC1521 20120803 MSFT 97) (XEN) ACPI: MCFG BF7904B0, 003C (r1 080312 OEMMCFG 20120803 MSFT 97) (XEN) ACPI: SLIT BF7904F0, 0030 (r1 080312 OEMSLIT 20120803 MSFT 97) (XEN) ACPI: OEMB BF79E040, 0086 (r1 080312 OEMB1521 20120803 MSFT 97) (XEN) ACPI: SRAT BF79A6A0, 01D0 (r1 080312 OEMSRAT 1 INTL 1) (XEN) ACPI: HPET BF79A870, 0038 (r1 080312 OEMHPET 20120803 MSFT 97) (XEN) ACPI: DMAR BF79E0D0, 0130 (r1 AMI OEMDMAR 1 MSFT 97) (XEN) ACPI: SSDT BF7A1710, 0363 (r1 DpgPmm CpuPm 12 INTL 20051117) (XEN) ACPI: EINJ BF79A8B0, 0130 (r1 AMIER AMI_EINJ 20120803 MSFT 97) (XEN) ACPI: BERT BF79AA40, 0030 (r1 AMIER AMI_BERT 20120803 MSFT 97) (XEN) ACPI: ERST BF79AA70, 01B0 (r1 AMIER AMI_ERST 20120803 MSFT 97) (XEN) ACPI: HEST BF79AC20, 00A8 (r1 AMIER ABC_HEST 20120803 MSFT 97) (XEN) System RAM: 98295MB (100654180kB) (XEN) SRAT: PXM 0 -> APIC 00 -> Node 0 (XEN) SRAT: PXM 0 -> APIC 02 -> Node 0 (XEN) SRAT: PXM 0 -> APIC 04 -> Node 0 (XEN) SRAT: PXM 0 -> APIC 06 -> Node 0 (XEN) SRAT: PXM 0 -> APIC 01 -> Node 0 (XEN) SRAT: PXM 0 -> APIC 03 -> Node 0 (XEN) SRAT: PXM 0 -> APIC 05 -> Node 0 (XEN) SRAT: PXM 0 -> APIC 07 -> Node 0 (XEN) SRAT: PXM 1 -> APIC 10 -> Node 1 (XEN) SRAT: PXM 1 -> APIC 12 -> Node 1 (XEN) SRAT: PXM 1 -> APIC 14 -> Node 1 (XEN) SRAT: PXM 1 -> APIC 16 -> Node 1 (XEN) SRAT: PXM 1 -> APIC 11 -> Node 1 (XEN) SRAT: PXM 1 -> APIC 13 -> Node 1 (XEN) SRAT: PXM 1 -> APIC 15 -> Node 1 (XEN) SRAT: PXM 1 -> APIC 17 -> Node 1 (XEN) SRAT: Node 0 PXM 0 0-a0000 (XEN) SRAT: Node 0 PXM 0 100000-c0000000 (XEN) SRAT: Node 0 PXM 0 100000000-c40000000 (XEN) SRAT: Node 1 PXM 1 c40000000-1840000000 (XEN) NUMA: Allocated memnodemap from 183e1b7000 - 183e1d0000 (XEN) NUMA: Using 8 for the hash shift. (XEN) Domain heap initialised DMA width 32 bits (XEN) CPU Vendor: Intel, Family 6 (0x6), Model 26 (0x1a), Stepping 5 (raw 000106a5) (XEN) found SMP MP-table at 000ff780 (XEN) DMI present. (XEN) Using APIC driver default (XEN) ACPI: PM-Timer IO Port: 0x808 (32 bits) (XEN) ACPI: SLEEP INFO: pm1x_cnt[1:804,1:0], pm1x_evt[1:800,1:0] (XEN) ACPI: wakeup_vec[bf79e00c], vec_size[20] (XEN) ACPI: Local APIC address 0xfee00000 (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled) (XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled) (XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x04] enabled) (XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x06] enabled) (XEN) ACPI: LAPIC (acpi_id[0x05] lapic_id[0x10] enabled) (XEN) ACPI: LAPIC (acpi_id[0x06] lapic_id[0x12] enabled) (XEN) ACPI: LAPIC (acpi_id[0x07] lapic_id[0x14] enabled) (XEN) ACPI: LAPIC (acpi_id[0x08] lapic_id[0x16] enabled) (XEN) ACPI: LAPIC (acpi_id[0x09] lapic_id[0x01] enabled) (XEN) ACPI: LAPIC (acpi_id[0x0a] lapic_id[0x03] enabled) (XEN) ACPI: LAPIC (acpi_id[0x0b] lapic_id[0x05] enabled) (XEN) ACPI: LAPIC (acpi_id[0x0c] lapic_id[0x07] enabled) (XEN) ACPI: LAPIC (acpi_id[0x0d] lapic_id[0x11] enabled) (XEN) ACPI: LAPIC (acpi_id[0x0e] lapic_id[0x13] enabled) (XEN) ACPI: LAPIC (acpi_id[0x0f] lapic_id[0x15] enabled) (XEN) ACPI: LAPIC (acpi_id[0x10] lapic_id[0x17] enabled) (XEN) ACPI: LAPIC (acpi_id[0x11] lapic_id[0x90] disabled) (XEN) ACPI: LAPIC (acpi_id[0x12] lapic_id[0x91] disabled) (XEN) ACPI: LAPIC (acpi_id[0x13] lapic_id[0x92] disabled) (XEN) ACPI: LAPIC (acpi_id[0x14] lapic_id[0x93] disabled) (XEN) ACPI: LAPIC (acpi_id[0x15] lapic_id[0x94] disabled) (XEN) ACPI: LAPIC (acpi_id[0x16] lapic_id[0x95] disabled) (XEN) ACPI: LAPIC (acpi_id[0x17] lapic_id[0x96] disabled) (XEN) ACPI: LAPIC (acpi_id[0x18] lapic_id[0x97] disabled) (XEN) ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) (XEN) Overriding APIC driver with bigsmp (XEN) ACPI: IOAPIC (id[0x08] address[0xfec00000] gsi_base[0]) (XEN) IOAPIC[0]: apic_id 8, version 32, address 0xfec00000, GSI 0-23 (XEN) ACPI: IOAPIC (id[0x09] address[0xfec8a000] gsi_base[24]) (XEN) IOAPIC[1]: apic_id 9, version 32, address 0xfec8a000, GSI 24-47 (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) (XEN) ACPI: IRQ0 used by override. (XEN) ACPI: IRQ2 used by override. (XEN) ACPI: IRQ9 used by override. (XEN) Enabling APIC mode: Phys. Using 2 I/O APICs (XEN) ACPI: HPET id: 0x8086a301 base: 0xfed00000 (XEN) PCI: MCFG configuration 0: base e0000000 segment 0000 buses 00 - ff (XEN) PCI: MCFG area at e0000000 reserved in E820 (XEN) PCI: Using MCFG for segment 0000 bus 00-ff (XEN) Failed to get Error Log Address Range. (XEN) HEST: Table parsing has been initialized (XEN) Using ACPI (MADT) for SMP configuration information (XEN) SMP: Allowing 24 CPUs (8 hotplug CPUs) (XEN) IRQ limits: 48 GSI, 3040 MSI/MSI-X (XEN) CPU0: Intel machine check reporting enabled (XEN) Speculative mitigation facilities: (XEN) Hardware features: (XEN) Compiled-in support: SHADOW_PAGING (XEN) Xen settings: BTI-Thunk N/A, SPEC_CTRL: No, Other: (XEN) L1TF: believed vulnerable, maxphysaddr L1D 44, CPUID 40, Safe address 10000000000 (XEN) Support for HVM VMs: RSB EAGER_FPU (XEN) Support for PV VMs: RSB EAGER_FPU (XEN) XPTI (64-bit PV only): Dom0 enabled, DomU enabled (without PCID) (XEN) PV L1TF shadowing: Dom0 disabled, DomU enabled (XEN) Using scheduler: SMP Credit Scheduler rev2 (credit2) (XEN) Initializing Credit2 scheduler (XEN) load_precision_shift: 18 (XEN) load_window_shift: 30 (XEN) underload_balance_tolerance: 0 (XEN) overload_balance_tolerance: -3 (XEN) runqueues arrangement: socket (XEN) cap enforcement granularity: 10ms (XEN) load tracking window length 1073741824 ns (XEN) Adding cpu 0 to runqueue 0 (XEN) First cpu on runqueue, activating (XEN) Platform timer is 14.318MHz HPET (XEN) Detected 2266.757 MHz processor. (XEN) Initing memory sharing. (XEN) alt table ffff82d0804309b8 -> ffff82d0804318d0 (XEN) Intel VT-d iommu 0 supported page sizes: 4kB. (XEN) Intel VT-d Snoop Control enabled. (XEN) Intel VT-d Dom0 DMA Passthrough not enabled. (XEN) Intel VT-d Queued Invalidation enabled. (XEN) Intel VT-d Interrupt Remapping enabled. (XEN) Intel VT-d Posted Interrupt not enabled. (XEN) Intel VT-d Shared EPT tables not enabled. (XEN) I/O virtualisation enabled (XEN) - Dom0 mode: Relaxed (XEN) Interrupt remapping enabled (XEN) nr_sockets: 3 (XEN) ENABLING IO-APIC IRQs (XEN) -> Using new ACK method (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1 (XEN) Allocated console ring of 128 KiB. (XEN) mwait-idle: MWAIT substates: 0x1120 (XEN) mwait-idle: v0.4.1 model 0x1a (XEN) mwait-idle: lapic_timer_reliable_states 0x2 (XEN) HPET: 0 timers usable for broadcast (4 total) (XEN) VMX: Supported advanced features: (XEN) - APIC MMIO access virtualisation (XEN) - APIC TPR shadow (XEN) - Extended Page Tables (EPT) (XEN) - Virtual-Processor Identifiers (VPID) (XEN) - Virtual NMI (XEN) - MSR direct-access bitmap (XEN) HVM: ASIDs enabled. (XEN) HVM: VMX enabled (XEN) HVM: Hardware Assisted Paging (HAP) detected (XEN) HVM: HAP page sizes: 4kB, 2MB (XEN) Adding cpu 1 to runqueue 0 (XEN) Adding cpu 2 to runqueue 0 (XEN) Adding cpu 3 to runqueue 0 (XEN) Adding cpu 4 to runqueue 0 (XEN) Adding cpu 5 to runqueue 0 (XEN) Adding cpu 6 to runqueue 0 (XEN) Adding cpu 7 to runqueue 0 (XEN) Adding cpu 8 to runqueue 1 (XEN) First cpu on runqueue, activating (XEN) Adding cpu 9 to runqueue 1 (XEN) Adding cpu 10 to runqueue 1 (XEN) Adding cpu 11 to runqueue 1 (XEN) Adding cpu 12 to runqueue 1 (XEN) Adding cpu 13 to runqueue 1 (XEN) Adding cpu 14 to runqueue 1 (XEN) Adding cpu 15 to runqueue 1 (XEN) Brought up 16 CPUs (XEN) build-id: aba2ec2f0b48776dac7a6e615962fec7240f6447 (XEN) ACPI sleep modes: S3 (XEN) VPMU: disabled (XEN) mcheck_poll: Machine check polling timer started. (XEN) Dom0 has maximum 816 PIRQs (XEN) NX (Execute Disable) protection active (XEN) *** Building a PVH Dom0 *** (XEN) Masked UR signaling on 0000:00:00.0 (XEN) Masked UR signaling on 0000:00:01.0 (XEN) Masked UR signaling on 0000:00:03.0 (XEN) Masked UR signaling on 0000:00:05.0 (XEN) Masked UR signaling on 0000:00:07.0 (XEN) Masked UR signaling on 0000:00:09.0 (XEN) Masked VT-d error signaling on 0000:00:14.0 (XEN) WARNING: PVH is an experimental mode with limited functionality (XEN) Initial low memory virq threshold set at 0x4000 pages. (XEN) Scrubbing Free RAM in background (XEN) Std. Loglevel: All (XEN) Guest Loglevel: All (XEN) *************************************************** (XEN) Booted on L1TF-vulnerable hardware with SMT/Hyperthreading (XEN) enabled. Please assess your configuration and choose an (XEN) explicit 'smt=' setting. See XSA-273. (XEN) *************************************************** (XEN) Booted on MLPDS/MFBDS-vulnerable hardware with SMT/Hyperthreading (XEN) enabled. Mitigations will not be fully effective. Please (XEN) choose an explicit smt= setting. See XSA-297. (XEN) *************************************************** (XEN) 3... 2... 1... (XEN) Xen is relinquishing VGA console. (XEN) *** Serial input to DOM0 (type 'CTRL-a' three times to switch input) (XEN) Freed 500kB init memory ---<>--- Copyright (c) 1992-2018 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 12.0-RELEASE-p10 GENERIC amd64 FreeBSD clang version 6.0.1 (tags/RELEASE_601/final 335540) (based on LLVM 6.0.1) VT(vga): text 80x25 XEN: Hypervisor version 4.12 detected. CPU: Intel(R) Xeon(R) CPU E5520 @ 2.27GHz (2266.76-MHz K8-class CPU) Origin="GenuineIntel" Id=0x106a5 Family=0x6 Model=0x1a Stepping=5 Features=0x1fc3fbff Features2=0x80982201 AMD Features=0x28100800 AMD Features2=0x1 TSC: P-state invariant Hypervisor: Origin = "XenVMMXenVMM" real memory = 5378043904 (5128 MB) avail memory = 4099301376 (3909 MB) Event timer "LAPIC" quality 100 ACPI APIC Table: <080312 APIC1521> FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs FreeBSD/SMP: 1 package(s) x 4 core(s) random: unblocking device. Firmware Warning (ACPI): 32/64X length mismatch in FADT/Gpe0Block: 128/64 (20181003/tbfadt-748) ioapic1: Changing APIC ID to 9 ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-24 on motherboard Launching APs: 2 1 3 random: entropy device external interface kbd1 at kbdmux0 netmap: loaded module [ath_hal] loaded module_register_init: MOD_LOAD (vesa, 0xffffffff810f9a00, 0) error 19 nexus0 vtvga0: on motherboard cryptosoft0: on motherboard acpi0: on motherboard acpi0: Power Button (fixed) cpu0: on acpi0 attimer0: port 0x40-0x43 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 atrtc0: port 0x70-0x71 irq 8 on acpi0 atrtc0: registered as a time-of-day clock, resolution 1.000000s Event timer "RTC" frequency 32768 Hz quality 0 hpet0: iomem 0xfed03000-0xfed033ff on acpi0 hpet0: HPET never increments, disabling device_attach: hpet0 attach returned 6 Timecounter "ACPI-safe" frequency 3579545 Hz quality 850 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: at device 1.0 on pci0 pci1: on pcib1 igb0: port 0xdc00-0xdc1f mem 0xfbce0000-0xfbcfffff,0xfbcc0000-0xfbcdffff,0xfbc9c000-0xfbc9ffff irq 28 at device 0.0 on pci1 igb0: attach_pre capping queues at 8 igb0: using 1024 tx descriptors and 1024 rx descriptors igb0: msix_init qsets capped at 8 igb0: pxm cpus: 4 queue msgs: 9 admincnt: 1 igb0: using 4 rx queues 4 tx queues igb0: Using MSIX interrupts with 5 vectors igb0: allocated for 4 tx_queues igb0: allocated for 4 rx_queues igb0: Ethernet address: 00:25:90:36:fd:2e igb0: netmap queues/slots: TX 4/1024, RX 4/1024 igb1: port 0xd880-0xd89f mem 0xfbc60000-0xfbc7ffff,0xfbc40000-0xfbc5ffff,0xfbc1c000-0xfbc1ffff irq 40 at device 0.1 on pci1 igb1: attach_pre capping queues at 8 igb1: using 1024 tx descriptors and 1024 rx descriptors igb1: msix_init qsets capped at 8 igb1: pxm cpus: 4 queue msgs: 9 admincnt: 1 igb1: using 4 rx queues 4 tx queues igb1: Using MSIX interrupts with 5 vectors igb1: allocated for 4 tx_queues igb1: allocated for 4 rx_queues igb1: Ethernet address: 00:25:90:36:fd:2f igb1: netmap queues/slots: TX 4/1024, RX 4/1024 pcib2: at device 3.0 on pci0 pci2: on pcib2 pcib3: at device 5.0 on pci0 pci3: on pcib3 igb2: port 0xec00-0xec1f mem 0xfbde0000-0xfbdfffff,0xfbdc0000-0xfbddffff,0xfbd9c000-0xfbd9ffff irq 26 at device 0.0 on pci3 igb2: attach_pre capping queues at 8 igb2: using 1024 tx descriptors and 1024 rx descriptors igb2: msix_init qsets capped at 8 igb2: pxm cpus: 4 queue msgs: 9 admincnt: 1 igb2: using 4 rx queues 4 tx queues igb2: Using MSIX interrupts with 5 vectors igb2: allocated for 4 tx_queues igb2: allocated for 4 rx_queues igb2: Ethernet address: 00:25:90:09:c8:da igb2: netmap queues/slots: TX 4/1024, RX 4/1024 igb3: port 0xe880-0xe89f mem 0xfbd60000-0xfbd7ffff,0xfbd40000-0xfbd5ffff,0xfbd1c000-0xfbd1ffff irq 25 at device 0.1 on pci3 igb3: attach_pre capping queues at 8 igb3: using 1024 tx descriptors and 1024 rx descriptors igb3: msix_init qsets capped at 8 igb3: pxm cpus: 4 queue msgs: 9 admincnt: 1 igb3: using 4 rx queues 4 tx queues igb3: Using MSIX interrupts with 5 vectors igb3: allocated for 4 tx_queues igb3: allocated for 4 rx_queues igb3: Ethernet address: 00:25:90:09:c8:db igb3: netmap queues/slots: TX 4/1024, RX 4/1024 pcib4: at device 7.0 on pci0 pci4: on pcib4 pcib5: at device 9.0 on pci0 pci5: on pcib5 pci0: at device 20.0 (no driver attached) pci0: at device 20.1 (no driver attached) pci0: at device 20.2 (no driver attached) pci0: at device 20.3 (no driver attached) uhci0: port 0xcc00-0xcc1f irq 16 at device 26.0 on pci0 uhci0: LegSup = 0x2f00 usbus0 on uhci0 usbus0: 12Mbps Full Speed USB v1.0 uhci1: port 0xc800-0xc81f irq 19 at device 26.2 on pci0 uhci2: LegSup = 0x2f00 usbus2 on uhci2 usbus2: 12Mbps Full Speed USB v1.0 ehci0: mem 0xfbeda000-0xfbeda3ff irq 18 at device 26.7 on pci0 usbus3: EHCI version 1.0 usbus3 on ehci0 usbus3: 480Mbps High Speed USB v2.0 uhci3: port 0xc480-0xc49f irq 23 at device 29.0 on pci0 uhci3: LegSup = 0x2f00 usbus4 on uhci3 usbus4: 12Mbps Full Speed USB v1.0 uhci4: port 0xc400-0xc41f irq 19 at device 29.1 on pci0 uhci4: LegSup = 0x2f00 usbus5 on uhci4 usbus5: 12Mbps Full Speed USB v1.0 uhci5: port 0xc080-0xc09f irq 18 at device 29.2 on pci0 uhci5: LegSup = 0x2f00 usbus6 on uhci5 usbus6: 12Mbps Full Speed USB v1.0 ehci1: mem 0xfbed8000-0xfbed83ff irq 23 at device 29.7 on pci0 usbus7: EHCI version 1.0 usbus7 on ehci1 usbus7: 480Mbps High Speed USB v2.0 pcib6: at device 30.0 on pci0 pci6: on pcib6 vgapci0: mem 0xf9000000-0xf9ffffff,0xfaffc000-0xfaffffff,0xfb000000-0xfb7fffff irq 18 at device 1.0 on pci6 vgapci0: Boot video device isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0xc000-0xc007,0xbc00-0xbc03,0xb880-0xb887,0xb800-0xb803,0xb480-0xb48f,0xb400-0xb40f irq 19 at device 31.2 on pci0 ata2: at channel 0 on atapci0 ata3: at channel 1 on atapci0 atapci1: port 0xb000-0xb007,0xac00-0xac03,0xa880-0xa887,0xa800-0xa803,0xa480-0xa48f,0xa400-0xa40f irq 19 at device 31.5 on pci0 ata4: at channel 0 on atapci1 ata5: at channel 1 on atapci1 acpi_button0: on acpi0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 xenpv0: on motherboard granttable0: on xenpv0 xen_et0: on xenpv0 Event timer "XENTIMER" frequency 1000000000 Hz quality 950 Timecounter "XENTIMER" frequency 1000000000 Hz quality 950 xen_et0: registered as a time-of-day clock, resolution 0.000001s xenstore0: on xenpv0 xsd_dev0: on xenpv0 evtchn0: on xenpv0 privcmd0: on xenpv0 gntdev0: on xenpv0 debug0: on xenpv0 xc0: on xenpv0 ipmi0: on isa0 ipmi0: KCS mode found at io 0xca2 alignment 0x1 on isa orm0: at iomem 0xc0000-0xc7fff,0xc8000-0xc8fff,0xc9000-0xc9fff,0xca000-0xcafff pnpid ORM0000 on isa0 vga0: at port 0x3d0-0x3db iomem 0xb8000-0xbffff pnpid PNP0900 on isa0 atkbdc0: at port 0x60,0x64 on isa0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] atkbdc0: non-PNP ISA device will be removed from GENERIC in FreeBSD 12. Timecounters tick every 10.000 msec ipmi0: IPMI device rev. 1, firmware rev. 3.13, version 2.0, device support mask 0xbf ipmi0: Number of channels 2 ipmi0: Attached watchdog ipmi0: Establishing power cycle handler ugen7.1: at usbus7 ugen5.1: at usbus5 ugen2.1: at usbus2 ugen1.1: at usbus1 uhub0: on usbus7 uhub1: on usbus2 ugen4.1: at usbus4 uhub2: on usbus1 uhub3: on usbus4 ugen3.1: at usbus3 ugen6.1: at usbus6 uhub4: on usbus5 uhub5: on usbus6 uhub6: on usbus3 ugen0.1: at usbus0 uhub7: on usbus0 uhub2: 2 ports with 2 removable, self powered uhub4: 2 ports with 2 removable, self powered uhub1: 2 ports with 2 removable, self powered uhub3: 2 ports with 2 removable, self powered uhub5: 2 ports with 2 removable, self powered uhub7: 2 ports with 2 removable, self powered ada0 at ata2 bus 0 scbus0 target 0 lun 0 ada0: ACS-3 ATA SATA 3.x device ada0: Serial Number ZGY0KSAS ada0: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes) ada0: 3815447MB (7814037168 512 byte sectors) ada0: quirks=0x1<4K> ada1 at ata2 bus 0 scbus0 target 1 lun 0 ada1: ACS-3 ATA SATA 3.x device ada1: Serial Number ZGY0KS9B ada1: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes) ada1: 3815447MB (7814037168 512 byte sectors) ada1: quirks=0x1<4K> ada2 at ata3 bus 0 scbus1 target 0 lun 0 ada2: ACS-3 ATA SATA 3.x device ada2: Serial Number ZGY0KSC9 ada2: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes) ada2: 3815447MB (7814037168 512 byte sectors) ada2: quirks=0x1<4K> ada3 at ata3 bus 0 scbus1 target 1 lun 0 ada3: ACS-3 ATA SATA 3.x device ada3: Serial Number ZDH1SGEB ada3: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes) ada3: 3815447MB (7814037168 512 byte sectors) ada3: quirks=0x1<4K> Trying to mount root from ufs:/dev/gpt/gptroot [rw]... uhub0: 6 ports with 6 removable, self powered uhub6: 6 ports with 6 removable, self powered cd0 at ata4 bus 0 scbus2 target 0 lun 0 cd0: Removable CD-ROM SCSI device cd0: 150.000MB/s transfers (SATA 1.x, UDMA2, ATAPI 12bytes, PIO 8192bytes) cd0: Attempt to query device size failed: NOT READY, Medium not present - tray closed usb_alloc_device: set address 2 failed (USB_ERR_TIMEOUT, ignored) usbd_setup_device_deusbd_setup_device_desc: getting device descriptor at addr 2 failed, USB_ERR_TIMEOUT usbd_req_re_enumerate: addr=2, set address failed! (USB_ERR_TIMEOUT, ignored) usbd_setup_device_deusbd_setup_device_desc: getting device descriptor at addr 2 failed, USB_ERR_TIMEOUT usbd_req_re_enumerate: addr=2, set address failed! (USB_ERR_TIMEOUT, ignored) usbd_setup_device_desc: getting device descriptor at addr 2 failed, USB_ERR_TIMEOUT ugen1.2: at usbus1 (disconnected) uhub_reattach_port: could nGEOM_MIRROR: Device mirror/gm0 launched (1/2). GEOM_MIRROR: Device gm0: rebuilding provider ada1. WARNING: /mnt was not properly dismounted !. !. 6  load: 0.09 cmd: sh 29 [biord] 557.49r 0.00u 0.01s 0% 2544k [Timed out and hit ctrl-a thrice to see what xen was doing] (XEN) *** Serial input to Xen (type 'CTRL-a' three times to switch input) (XEN) 'h' pressed -> showing installed handlers (XEN) key '%' (ascii '25') => trap to xendbg (XEN) key '*' (ascii '2a') => print all diagnostics (XEN) key '+' (ascii '2b') => increase log level threshold (XEN) key '-' (ascii '2d') => decrease log level threshold (XEN) key '0' (ascii '30') => dump Dom0 registers (XEN) key 'A' (ascii '41') => toggle alternative key handling (XEN) key 'D' (ascii '44') => dump VT-x EPT tables (XEN) key 'G' (ascii '47') => toggle host/guest log level adjustment (XEN) key 'H' (ascii '48') => dump heap info (XEN) key 'I' (ascii '49') => dump HVM irq info (XEN) key 'M' (ascii '4d') => dump MSI state (XEN) key 'N' (ascii '4e') => trigger an NMI (XEN) key 'Q' (ascii '51') => dump PCI devices (XEN) key 'R' (ascii '52') => reboot machine (XEN) key 'V' (ascii '56') => dump iommu info (XEN) key 'a' (ascii '61') => dump timer queues (XEN) key 'c' (ascii '63') => dump ACPI Cx structures (XEN) key 'd' (ascii '64') => dump registers (XEN) key 'e' (ascii '65') => dump evtchn info (XEN) key 'g' (ascii '67') => print grant table usage (XEN) key 'h' (ascii '68') => show this message (XEN) key 'i' (ascii '69') => dump interrupt bindings (XEN) key 'm' (ascii '6d') => memory info (XEN) key 'n' (ascii '6e') => NMI statistics (XEN) key 'o' (ascii '6f') => dump iommu p2m table (XEN) key 'q' (ascii '71') => dump domain (and guest debug) info (XEN) key 'r' (ascii '72') => dump run queues (XEN) key 's' (ascii '73') => dump softtsc stats (XEN) key 't' (ascii '74') => display multi-cpu clock info (XEN) key 'u' (ascii '75') => dump NUMA info (XEN) key 'v' (ascii '76') => dump VT-x VMCSs (XEN) key 'w' (ascii '77') => synchronously dump console ring buffer (dmesg) (XEN) key 'x' (ascii '78') => print livepatch info (XEN) key 'z' (ascii '7a') => dump IOAPIC info (XEN) 'e' pressed -> dumping event-channel info (XEN) Event channel information for domain 0: (XEN) Polling vCPUs: {} (XEN) port [p/m/s] (XEN) 1 [0/0/0]: s=6 n=0 x=0 (XEN) 2 [0/0/0]: s=6 n=0 x=0 (XEN) 3 [0/0/0]: s=6 n=0 x=0 (XEN) 4 [0/0/0]: s=6 n=0 x=0 (XEN) 5 [0/0/0]: s=6 n=0 x=0 (XEN) 6 [0/0/0]: s=6 n=0 x=0 (XEN) 7 [0/0/0]: s=6 n=0 x=0 (XEN) 8 [0/0/0]: s=6 n=0 x=0 (XEN) 9 [0/0/0]: s=6 n=0 x=0 (XEN) 10 [0/0/0]: s=6 n=1 x=0 (XEN) 11 [0/0/0]: s=6 n=1 x=0 (XEN) 12 [0/0/0]: s=6 n=1 x=0 (XEN) 13 [0/0/0]: s=6 n=1 x=0 (XEN) 14 [0/0/0]: s=6 n=1 x=0 (XEN) 15 [0/0/0]: s=6 n=1 x=0 (XEN) 16 [0/0/0]: s=6 n=1 x=0 (XEN) 17 [0/0/0]: s=6 n=1 x=0 (XEN) 18 [0/0/0]: s=6 n=1 x=0 (XEN) 19 [0/0/0]: s=6 n=2 x=0 (XEN) 20 [0/0/0]: s=6 n=2 x=0 (XEN) 21 [0/0/0]: s=6 n=2 x=0 (XEN) 22 [0/0/0]: s=6 n=2 x=0 (XEN) 23 [0/0/0]: s=6 n=2 x=0 (XEN) 24 [0/0/0]: s=6 n=2 x=0 (XEN) 25 [0/0/0]: s=6 n=2 x=0 (XEN) 26 [0/0/0]: s=6 n=2 x=0 (XEN) 27 [0/0/0]: s=6 n=2 x=0 (XEN) 28 [0/0/0]: s=6 n=3 x=0 (XEN) 29 [0/0/0]: s=6 n=3 x=0 (XEN) 30 [0/0/0]: s=6 n=3 x=0 (XEN) 31 [0/0/0]: s=6 n=3 x=0 (XEN) 32 [0/0/0]: s=6 n=3 x=0 (XEN) 33 [0/0/0]: s=6 n=3 x=0 (XEN) 34 [0/0/0]: s=6 n=3 x=0 (XEN) 35 [0/0/0]: s=6 n=3 x=0 (XEN) 36 [0/0/0]: s=6 n=3 x=0 (XEN) 37 [0/0/0]: s=5 n=0 x=0 v=0 (XEN) 38 [0/0/0]: s=5 n=1 x=0 v=0 (XEN) 39 [0/0/0]: s=5 n=2 x=0 v=0 (XEN) 40 [0/0/0]: s=5 n=3 x=0 v=0 (XEN) 41 [0/0/0]: s=2 n=1 x=0 d=0 (XEN) 42 [0/0/0]: s=5 n=0 x=0 v=1 (XEN) 43 [0/0/0]: s=5 n=1 x=0 v=1 (XEN) 44 [0/0/0]: s=5 n=2 x=0 v=1 (XEN) 45 [0/0/0]: s=5 n=3 x=0 v=1 (XEN) 46 [0/0/0]: s=5 n=0 x=0 v=2 (XEN) *** Serial input to DOM0 (type 'CTRL-a' three times to switch input) load: 0.25 cmd: rcorder 34 [biord] 222.28r 0.00u 0.00s 0% 2024k load: 0.25 cmd: rcorder 34 [biord] 240.94r 0.00u 0.00s 0% 2024k load: 0.08 cmd: rcorder 34 [biord] 513.71r 0.00u 0.00s 0% 2040k Setting hostuuid: 49434d53-0200-9009-2500-09902500dac8. Setting hostid: 0x0535156f. ZFS NOTICE: Prefetch is disabled by default if less than 4GB of RAM is present; to enable, add "vfs.zfs.prefetch_disable=0" to /boot/loader.conf. ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Starting file system checks: load: 0.34 cmd: fsck 61 [biord] 7.62r 0.00u 0.00s 0% 1968k /dev/gpt/gptroot: DEFER FOR BACKGROUND CHECKING Script /etc/rc.d/fsck running WARNING: / was not properly dismounted Mounting local filesystems:. load: 0.12 cmd: find 92 [biord] 61.82r 0.00u 0.00s 0% 2040k Script /etc/rc.d/cleanvar running load: 0.18 cmd: sh 97 [biord] 104.79r 0.00u 0.00s 0% 2648k Script /etc/rc.d/ldconfig running ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/gcc9 /usr/local/lib/perl5/5.30/mach/CORE 32-bit compatibility ldconfig path: /usr/lib32 Setting hostname: xen3.via.net. Setting up harvesting: [UMA],[FS_ATIME],SWI,INTERRUPT,NET_NG,NET_ETHER,NET_TUN,MOUSE,KEYBOARD,ATTACH,CACHED Feeding entropy: . load: 0.05 cmd: sed 171 [biord] 17.05r 0.00u 0.00s 0% 2020k Script /etc/rc.d/netif running bridge0: Ethernet address: 02:05:35:15:6f:00 Created clone interfaces: bridge0. lo0: link state changed to UP ifconfig: BRDGADD vlan34: No such file or directory igb0: link state changed to UP igb1: link state changed to UP Starting Network: lo0 igb0 igb1 igb2 igb3 bridge0. lo0: flags=8049 metric 0 mtu 16384 options=680003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5 inet 127.0.0.1 netmask 0xff000000 groups: lo nd6 options=21 igb0: flags=8843 metric 0 mtu 1500 options=e527bb ether 00:25:90:36:fd:2e inet 10.0.0.202 netmask 0xffffff00 broadcast 10.0.0.255 media: Ethernet autoselect (1000baseT ) status: active nd6 options=29 igb1: flags=8843 metric 0 mtu 1500 options=e527bb ether 00:25:90:36:fd:2f media: Ethernet autoselect (1000baseT ) status: active nd6 options=29 igb2: flags=8802 metric 0 mtu 1500 options=e527bb ether 00:25:90:09:c8:da media: Ethernet autoselect status: no carrier nd6 options=29 igb3: flags=8802 metric 0 mtu 1500 options=e527bb ether 00:25:90:09:c8:db media: Ethernet autoselect status: no carrier nd6 options=29 bridge0: flags=8843 metric 0 mtu 1500 ether 02:05:35:15:6f:00 id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 groups: bridge nd6 options=9 Starting devd. load: 0.33 cmd: devd 462 [biord] 105.15r 0.00u 0.00s 0% 516k Script /etc/rc.d/devd running add host 127.0.0.1: gateway lo0 fib 0: route already in table add net default: gateway 10.0.0.1 add host ::1: gateway lo0 fib 0: route already in table add net fe80::: gateway ::1 add net ff02::: gateway ::1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 Creating and/or trimming log files. Starting syslogd. syslogd: timed out waiting for child /etc/rc: WARNING: failed to start syslogd Setting date via ntp. ¿€¹10 Oct 13:58:35 ntpdate[563]: Can't find host clock.via.net: hostname nor servname provided, or not known (8) 10 Oct 13:58:35 ntpdate[563]: no servers can be used, exiting Clearing /tmp (X related). Cleaning xenstore database. Starting xenservices: xenstored, xenconsoled.Oct 10 14:05:06 xen3 xenstored[689]: Checking store ... xs_dev0: on xenstore0 xenbusb_front0: on xenstore0 xenbusb_back0: on xenstore0 xenballoon0: on xenstore0 xctrl0: on xenstore0 Oct 10 14:05:06 xen3 xenstored[689]: Checking store complete. Setting domain 0 name, domid and JSON config... Done setting up Dom0 load: 0.08 cmd: utx 701 [biord] 6.48r 0.00u 0.00s 0% 2096k Script /etc/rc.d/utx running Updating motd: load: 0.07 cmd: mktemp 713 [biord] 7.77r 0.00u 0.00s 0% 2108k Script /etc/rc.d/motd running . Mounting late filesystems:. Configuring vt: blanktime. Performing sanity check on sshd configuration. load: 0.16 cmd: sshd 757 [biord] 362.18r 0.00u 0.00s 0% 4376k Script /etc/rc.d/sshd running Starting sshd. Starting sendmail_submit. load: 0.25 cmd: sendmail 761 [select] 236.13r 0.00u 0.00s 0% 5908k jScript /etc/rc.d/sendmail running Starting sendmail_msp_queue. Starting cron. Autoloading module: ioat.ko ioat0: mem 0xfbef8000-0xfbefbfff irq 43 at device 22.0 on pci0 ioat0: Capabilities: 77 ioat1: mem 0xfbef4000-0xfbef7fff irq 44 at device 22.1 on pci0 ioat1: Capabilities: 77 ioat2: mem 0xfbef0000-0xfbef3fff irq 45 at device 22.2 on pci0 ioat2: Capabilities: 77 ioat3: mem 0xfbeec000-0xfbeeffff irq 46 at device 22.3 on pci0 ioat3: Capabilities: 77 ioat4: mem 0xfbee8000-0xfbeebfff irq 43 at device 22.4 on pci0 ioat4: Capabilities: 77 ioat5: mem 0xfbee4000-0xfbee7fff irq 44 at device 22.5 on pci0 ioat5: Capabilities: 77 ioat6: mem 0xfbee0000-0xfbee3fff irq 45 at device 22.6 on pci0 ioat6: Capabilities: 77 ioat7: mem 0xfbedc000-0xfbedffff irq 46 at device 22.7 on pci0 ioat7: Capabilities: 77 Starting background file system checks in 60 seconds. Thu Oct 10 14:41:59 PDT 2019 Oct 10 14:43:05 xen3 getty[843]: open /dev/ttyu1: No such file or directory Oct 10 14:43:05 xen3 getty[845]: open /dev/ttyu1: No such file or directory Oct 10 14:43:05 xen3 getty[846]: open /dev/ttyu1: No such file or directory Oct 10 14:43:05 xen3 getty[847]: open /dev/ttyu1: No such file or directory Oct 10 14:43:05 xen3 getty[848]: open /dev/ttyu1: No such file or directory Oct 10 14:43:05 xen3 init[849]: getty repeating too quickly on port /dev/ttyu1, sleeping 30 secs Oct 10 14:43:41 xen3 getty[849]: open /dev/ttyu1: No such file or directory Oct 10 14:44:56 xen3 getty[864]: open /dev/ttyu1: No such file or directory Oct 10 14:44:56 xen3 getty[865]: open /dev/ttyu1: No such file or directory Oct 10 14:44:56 xen3 init[866]: getty repeating too quickly on port /dev/ttyu1, sleeping 30 secs From owner-freebsd-xen@freebsd.org Fri Oct 11 07:06:36 2019 Return-Path: Delivered-To: freebsd-xen@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 323AE13FF82 for ; Fri, 11 Oct 2019 07:06:36 +0000 (UTC) (envelope-from roger.pau@citrix.com) Received: from esa6.hc3370-68.iphmx.com (esa6.hc3370-68.iphmx.com [216.71.155.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "esa6.hc3370-68.iphmx.com", Issuer "HydrantID SSL ICA G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46qJsQ3JyCz458T for ; Fri, 11 Oct 2019 07:06:34 +0000 (UTC) (envelope-from roger.pau@citrix.com) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1570777594; h=date:from:to:cc:subject:message-id:references: mime-version:in-reply-to; bh=/EG7RCAISkEAcnXgqN+CPCI/ZyRyVj4JCeeIBbqVTuM=; b=N8c5ZoabAwVp1Zqiz/vnx1oeqKKnokGP6/arJ1eojqf0oDR50pGhN3Cv uBNI/GZMowS2uUKcIgwCWJXJ/ffDRwCkXUcmXWgafRnfLV5eVT8n4pWUi OQlDIk5l+5OXMIlIf2eX0R1si5rXk+VK3LcDaUVVJeli148RLvpE5A+66 s=; Received-SPF: None (esa6.hc3370-68.iphmx.com: no sender authenticity information available from domain of roger.pau@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa6.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa6.hc3370-68.iphmx.com: domain of roger.pau@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa6.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa6.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa6.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: ByadXV3Vm57xfitQVD0rVs0WXHKFU78wjMWTUFg5eNJE0ESmeoJN5wZrK85Q/9wiJo6W3peeJB zOv6SOqHfBj02z+g85hWBOCG26XnHzbu5hCJMnXiaHuCOQHUYBsvCX+G3Hj8CaxgeVKKWUxQmd XiaOAj2bjU2MOQ0fQmjfPIJ1hj9KoUoHmiX44dOTEviQ8DeGRXP7My/Y9h6lRYWt3NF9CnMZGd M7kuv5tG0DdpT4SP1x114Py/ceT159sHo1qQJ2kCqaAANPGxtUZYh6+xkkcET7L/ULEDAS3a+R DAU= X-SBRS: 2.7 X-MesageID: 7066010 X-Ironport-Server: esa6.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.67,283,1566878400"; d="scan'208";a="7066010" Date: Fri, 11 Oct 2019 09:06:21 +0200 From: Roger Pau =?iso-8859-1?Q?Monn=E9?= To: Brian Buhrow CC: Subject: Re: i/o is very slow on FreeBSD dom0 with Xen-4.12 and Freebsd-12 Message-ID: <20191011070621.GA1004@Air-de-Roger> References: <201910102323.x9ANNACe003999@lothlorien.nfbcal.org> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: <201910102323.x9ANNACe003999@lothlorien.nfbcal.org> User-Agent: Mutt/1.12.2 (2019-09-21) X-ClientProxiedBy: AMSPEX02CAS01.citrite.net (10.69.22.112) To AMSPEX02CL02.citrite.net (10.69.22.126) X-Rspamd-Queue-Id: 46qJsQ3JyCz458T X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=citrix.com header.s=securemail header.b=N8c5Zoab; dmarc=pass (policy=none) header.from=citrix.com; spf=pass (mx1.freebsd.org: domain of roger.pau@citrix.com designates 216.71.155.175 as permitted sender) smtp.mailfrom=roger.pau@citrix.com X-Spamd-Result: default: False [-5.44 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[citrix.com:s=securemail]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; RCVD_DKIM_ARC_DNSWL_MED(-0.50)[]; DWL_DNSWL_LOW(-1.00)[citrix.com.dwl.dnswl.org : 127.0.9.1]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_MED(-0.20)[175.155.71.216.list.dnswl.org : 127.0.3.2]; DKIM_TRACE(0.00)[citrix.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[citrix.com,none]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MID_RHS_NOT_FQDN(0.50)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:16417, ipnet:216.71.154.0/23, country:US]; IP_SCORE(-1.24)[ip: (-2.33), ipnet: 216.71.154.0/23(-2.55), asn: 16417(-1.27), country: US(-0.05)] X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Oct 2019 07:06:36 -0000 On Thu, Oct 10, 2019 at 04:23:10PM -0700, Brian Buhrow wrote: > hello. I'm trying to get a FreeBSD-12 xen server up and running and > have run into the issue that i/o seems to be very slow for the dom0, which > is FreeBSD-12.0-stable, and Xen is xen-4.12. > I followed the instructions for making FreeBSD a dom0 as written in > 21.8 of the FreeBSD handbook. All went well until I rebooted the machine > with the /boot/xen kern loading. The dom0 boots, but takes about 2 hours > to get to the login prompt and, on the console, when ever I press > control-t, it says what ever process is currently running is in biord. > I'm familiar with NetBSD and how to make it run xen, and am familiar with > FreeBSD as well. Stil, I'm sure I'm doing something wrong that's simple, > or, I missed some instruction that will seem obvious in retrospect. > If anyone can help me shed light on the trouble, that would be great! Thanks for testing it out. What you do is correct, I think there's either an issue in Xen or the firmware. > Below are my /boot/loader.conf and a transcript of a sample boot. > Note that it took about 2 hours to get from the initial loading of the > kernel to the getty error messages. > -thanks > -Brian > > > boot_serial="YES" # -h: Use serial console > comconsole_speed="115200" # Set the current serial console speed > comconsole_port="0x2f8" # Set the current serial console port > #console="vidconsole" # A comma separated list of console(s) > console="comconsole" # A comma separated list of console(s) > geom_mirror_load="YES" # RAID1 disk driver (see gmirror(8)) > ipmi_load="YES" # Ripmi driver > if_tap_load="YES" # Load bridge driver. > if_vlan_load="YES" # Load vlan driver. > # Turn on Xen (BB 10/08/2019) > xen_kernel="/boot/xen" > xen_cmdline="dom0_mem=4096m dom0_max_vcpus=4 dom0=pvh console=com2,vga clocksource=hpet com2=115200,8n1 guest_loglvl=all loglvl=all" Any reason to force the clock source to HPET? Not that's it's wrong, I'm just curious about why you need that. Can you change dom0=pvh to dom0=pvh,verbose and also add iommu=verbose,debug and sync_console and post the boot log again. There's no need to wait for 2h, just post what you get up to the point where the box freezes. [...] > Trying to mount root from ufs:/dev/gpt/gptroot [rw]... > uhub0: 6 ports with 6 removable, self powered > uhub6: 6 ports with 6 removable, self powered > cd0 at ata4 bus 0 scbus2 target 0 lun 0 > cd0: Removable CD-ROM SCSI device > cd0: 150.000MB/s transfers (SATA 1.x, UDMA2, ATAPI 12bytes, PIO 8192bytes) > cd0: Attempt to query device size failed: NOT READY, Medium not present - tray closed > usb_alloc_device: set address 2 failed (USB_ERR_TIMEOUT, ignored) > usbd_setup_device_deusbd_setup_device_desc: getting device descriptor at addr 2 failed, USB_ERR_TIMEOUT > usbd_req_re_enumerate: addr=2, set address failed! (USB_ERR_TIMEOUT, ignored) > usbd_setup_device_deusbd_setup_device_desc: getting device descriptor at addr 2 failed, USB_ERR_TIMEOUT > usbd_req_re_enumerate: addr=2, set address failed! (USB_ERR_TIMEOUT, ignored) > usbd_setup_device_desc: getting device descriptor at addr 2 failed, USB_ERR_TIMEOUT > ugen1.2: at usbus1 (disconnected) > uhub_reattach_port: could nGEOM_MIRROR: Device mirror/gm0 launched (1/2). > GEOM_MIRROR: Device gm0: rebuilding provider ada1. > WARNING: /mnt was not properly dismounted The above timeouts point out that at least the usb controller is not working properly. Let's see if we can get more data with the new options on the command line. Thanks, Roger. From owner-freebsd-xen@freebsd.org Fri Oct 11 16:26:05 2019 Return-Path: Delivered-To: freebsd-xen@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B9CF0154E70 for ; Fri, 11 Oct 2019 16:26:05 +0000 (UTC) (envelope-from buhrow@lothlorien.nfbcal.org) Received: from lothlorien.nfbcal.org (ns.NFBCAL.ORG [157.22.230.125]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "nfbcal.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46qYH01jGBz3JQM for ; Fri, 11 Oct 2019 16:26:03 +0000 (UTC) (envelope-from buhrow@lothlorien.nfbcal.org) Received: from lothlorien.nfbcal.org (localhost [127.0.0.1]) by lothlorien.nfbcal.org (8.15.2/8.14.1-NFBNETBSD) with ESMTPS id x9BGPwdO028228 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 11 Oct 2019 09:25:58 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.97.3 at lothlorien.nfbcal.org Received: (from buhrow@localhost) by lothlorien.nfbcal.org (8.15.2/8.12.11) id x9BGPvot021852; Fri, 11 Oct 2019 09:25:57 -0700 (PDT) Message-Id: <201910111625.x9BGPvot021852@lothlorien.nfbcal.org> From: Brian Buhrow Date: Fri, 11 Oct 2019 09:25:57 -0700 In-Reply-To: <20191011070621.GA1004@Air-de-Roger> X-Mailer: Mail User's Shell (7.2.6 beta(4.pl1)+dynamic 20000103) To: Roger Pau =?iso-8859-1?Q?Monn=E9?= Subject: Re: i/o is very slow on FreeBSD dom0 with Xen-4.12 and Freebsd-12 Cc: , buhrow@nfbcal.org X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (lothlorien.nfbcal.org [127.0.0.1]); Fri, 11 Oct 2019 09:25:58 -0700 (PDT) X-Rspamd-Queue-Id: 46qYH01jGBz3JQM X-Spamd-Bar: ++ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of buhrow@lothlorien.nfbcal.org designates 157.22.230.125 as permitted sender) smtp.mailfrom=buhrow@lothlorien.nfbcal.org X-Spamd-Result: default: False [2.23 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.93)[-0.930,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+a:ns.nfbcal.org]; NEURAL_HAM_LONG(-0.33)[-0.327,0]; R_BAD_CTE_7BIT(3.50)[7bit]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[nfbcal.org]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; IP_SCORE(-0.01)[country: US(-0.05)]; FORGED_SENDER(0.30)[buhrow@nfbcal.org,buhrow@lothlorien.nfbcal.org]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:7091, ipnet:157.22.0.0/16, country:US]; FROM_NEQ_ENVFROM(0.00)[buhrow@nfbcal.org,buhrow@lothlorien.nfbcal.org]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Oct 2019 16:26:05 -0000 hello. Thanks for the response on this issue. I tried forcing the clock source to hpet0 because I initially thought my problem was that the dom0 wasn't getting a reliable clock source and therefore was getting "stuck" waiting for clock ticks. I've turned that switch off for the moment, since I don't think that's the problem. In any case, below is the output of the boot log up until the system becomes extremely slow. I hope this is a simple issue. -thanks -Brian boot_serial="YES" # -h: Use serial console comconsole_speed="115200" # Set the current serial console speed comconsole_port="0x2f8" # Set the current serial console port #console="vidconsole" # A comma separated list of console(s) console="comconsole" # A comma separated list of console(s) geom_mirror_load="YES" # RAID1 disk driver (see gmirror(8)) ipmi_load="YES" # Ripmi driver if_tap_load="YES" # Load bridge driver. if_vlan_load="YES" # Load vlan driver. # Turn on Xen (BB 10/08/2019) xen_kernel="/boot/xen" xen_cmdline="dom0_mem=4096m dom0_max_vcpus=4 dom0=pvh,verbose console=com2,vga iommu=verbose,debug sync_console=true com2=115200,8n1 guest_loglvl=all loglvl=all" Loading Xen kernel... boot/xen data=0x24a9b0+0x145650 Loading kernel... /boot/kernel/kernel | loading required module 'smbus' /boot/kernel/smbus.ko size 0x2e18 at 0x2427000 /boot/kernel/if_tap.ko size 0x82a0 at 0x242a000 /boot/kernel/geom_mirror.ko \size 0x27f48 at 0x2433000 /boot/entropy size=0x1000 Xen 4.12.1 (XEN) Xen version 4.12.1 (buhrow@) (FreeBSD clang version 6.0.1 (tags/RELEASE_601/final 335540) (based on LLVM 6.0.1)) debug=n Thu Oct 3 11:34:20 PDT 2019 (XEN) Latest ChangeSet: (XEN) Console output is synchronous. (XEN) Bootloader: FreeBSD Loader (XEN) Command line: dom0_mem=4096m dom0_max_vcpus=4 dom0=pvh,verbose console=com2,vga iommu=verbose,debug sync_console=true com2=115200,8n1 guest_loglvl=all loglvl=all (XEN) Xen image load base address: 0 (XEN) Video information: (XEN) VGA is text mode 80x25, font 8x16 (XEN) VBE/DDC methods: none; EDID transfer time: 0 seconds (XEN) EDID info not retrieved because no DDC retrieval method detected (XEN) Disc information: (XEN) Found 4 MBR signatures (XEN) Found 4 EDD information structures (XEN) Xen-e820 RAM map: (XEN) 0000000000000000 - 0000000000099800 (usable) (XEN) 0000000000099800 - 00000000000a0000 (reserved) (XEN) 00000000000e0000 - 0000000000100000 (reserved) (XEN) 0000000000100000 - 00000000bf780000 (usable) (XEN) 00000000bf78e000 - 00000000bf790000 type 9 (XEN) 00000000bf790000 - 00000000bf79e000 (ACPI data) (XEN) 00000000bf79e000 - 00000000bf7d0000 (ACPI NVS) (XEN) 00000000bf7d0000 - 00000000bf7e0000 (reserved) (XEN) 00000000bf7ec000 - 00000000c0000000 (reserved) (XEN) 00000000e0000000 - 00000000f0000000 (reserved) (XEN) 00000000fee00000 - 00000000fee01000 (reserved) (XEN) 00000000ffc00000 - 0000000100000000 (reserved) (XEN) 0000000100000000 - 0000001840000000 (usable) (XEN) New Xen image base address: 0xbf000000 (XEN) ACPI: RSDP 000FACE0, 0024 (r2 ACPIAM) (XEN) ACPI: XSDT BF790100, 008C (r1 SMCI 20120803 MSFT 97) (XEN) ACPI: FACP BF790290, 00F4 (r3 080312 FACP1521 20120803 MSFT 97) (XEN) ACPI: DSDT BF7906A0, 6580 (r1 10600 10600000 0 INTL 20051117) (XEN) ACPI: FACS BF79E000, 0040 (XEN) ACPI: APIC BF790390, 011E (r1 080312 APIC1521 20120803 MSFT 97) (XEN) ACPI: MCFG BF7904B0, 003C (r1 080312 OEMMCFG 20120803 MSFT 97) (XEN) ACPI: SLIT BF7904F0, 0030 (r1 080312 OEMSLIT 20120803 MSFT 97) (XEN) ACPI: OEMB BF79E040, 0086 (r1 080312 OEMB1521 20120803 MSFT 97) (XEN) ACPI: SRAT BF79A6A0, 01D0 (r1 080312 OEMSRAT 1 INTL 1) (XEN) ACPI: HPET BF79A870, 0038 (r1 080312 OEMHPET 20120803 MSFT 97) (XEN) ACPI: DMAR BF79E0D0, 0130 (r1 AMI OEMDMAR 1 MSFT 97) (XEN) ACPI: SSDT BF7A1710, 0363 (r1 DpgPmm CpuPm 12 INTL 20051117) (XEN) ACPI: EINJ BF79A8B0, 0130 (r1 AMIER AMI_EINJ 20120803 MSFT 97) (XEN) ACPI: BERT BF79AA40, 0030 (r1 AMIER AMI_BERT 20120803 MSFT 97) (XEN) ACPI: ERST BF79AA70, 01B0 (r1 AMIER AMI_ERST 20120803 MSFT 97) (XEN) ACPI: HEST BF79AC20, 00A8 (r1 AMIER ABC_HEST 20120803 MSFT 97) (XEN) System RAM: 98295MB (100654180kB) (XEN) SRAT: PXM 0 -> APIC 00 -> Node 0 (XEN) SRAT: PXM 0 -> APIC 02 -> Node 0 (XEN) SRAT: PXM 0 -> APIC 04 -> Node 0 (XEN) SRAT: PXM 0 -> APIC 06 -> Node 0 (XEN) SRAT: PXM 0 -> APIC 01 -> Node 0 (XEN) SRAT: PXM 0 -> APIC 03 -> Node 0 (XEN) SRAT: PXM 0 -> APIC 05 -> Node 0 (XEN) SRAT: PXM 0 -> APIC 07 -> Node 0 (XEN) SRAT: PXM 1 -> APIC 10 -> Node 1 (XEN) SRAT: PXM 1 -> APIC 12 -> Node 1 (XEN) SRAT: PXM 1 -> APIC 14 -> Node 1 (XEN) SRAT: PXM 1 -> APIC 16 -> Node 1 (XEN) SRAT: PXM 1 -> APIC 11 -> Node 1 (XEN) SRAT: PXM 1 -> APIC 13 -> Node 1 (XEN) SRAT: PXM 1 -> APIC 15 -> Node 1 (XEN) SRAT: PXM 1 -> APIC 17 -> Node 1 (XEN) SRAT: Node 0 PXM 0 0-a0000 (XEN) SRAT: Node 0 PXM 0 100000-c0000000 (XEN) SRAT: Node 0 PXM 0 100000000-c40000000 (XEN) SRAT: Node 1 PXM 1 c40000000-1840000000 (XEN) NUMA: Allocated memnodemap from 183e1b7000 - 183e1d0000 (XEN) NUMA: Using 8 for the hash shift. (XEN) Domain heap initialised DMA width 32 bits (XEN) CPU Vendor: Intel, Family 6 (0x6), Model 26 (0x1a), Stepping 5 (raw 000106a5) (XEN) found SMP MP-table at 000ff780 (XEN) DMI present. (XEN) Using APIC driver default (XEN) ACPI: PM-Timer IO Port: 0x808 (32 bits) (XEN) ACPI: SLEEP INFO: pm1x_cnt[1:804,1:0], pm1x_evt[1:800,1:0] (XEN) ACPI: wakeup_vec[bf79e00c], vec_size[20] (XEN) ACPI: Local APIC address 0xfee00000 (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled) (XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled) (XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x04] enabled) (XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x06] enabled) (XEN) ACPI: LAPIC (acpi_id[0x05] lapic_id[0x10] enabled) (XEN) ACPI: LAPIC (acpi_id[0x06] lapic_id[0x12] enabled) (XEN) ACPI: LAPIC (acpi_id[0x07] lapic_id[0x14] enabled) (XEN) ACPI: LAPIC (acpi_id[0x08] lapic_id[0x16] enabled) (XEN) ACPI: LAPIC (acpi_id[0x09] lapic_id[0x01] enabled) (XEN) ACPI: LAPIC (acpi_id[0x0a] lapic_id[0x03] enabled) (XEN) ACPI: LAPIC (acpi_id[0x0b] lapic_id[0x05] enabled) (XEN) ACPI: LAPIC (acpi_id[0x0c] lapic_id[0x07] enabled) (XEN) ACPI: LAPIC (acpi_id[0x0d] lapic_id[0x11] enabled) (XEN) ACPI: LAPIC (acpi_id[0x0e] lapic_id[0x13] enabled) (XEN) ACPI: LAPIC (acpi_id[0x0f] lapic_id[0x15] enabled) (XEN) ACPI: LAPIC (acpi_id[0x10] lapic_id[0x17] enabled) (XEN) ACPI: LAPIC (acpi_id[0x11] lapic_id[0x90] disabled) (XEN) ACPI: LAPIC (acpi_id[0x12] lapic_id[0x91] disabled) (XEN) ACPI: LAPIC (acpi_id[0x13] lapic_id[0x92] disabled) (XEN) ACPI: LAPIC (acpi_id[0x14] lapic_id[0x93] disabled) (XEN) ACPI: LAPIC (acpi_id[0x15] lapic_id[0x94] disabled) (XEN) ACPI: LAPIC (acpi_id[0x16] lapic_id[0x95] disabled) (XEN) ACPI: LAPIC (acpi_id[0x17] lapic_id[0x96] disabled) (XEN) ACPI: LAPIC (acpi_id[0x18] lapic_id[0x97] disabled) (XEN) ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) (XEN) Overriding APIC driver with bigsmp (XEN) ACPI: IOAPIC (id[0x08] address[0xfec00000] gsi_base[0]) (XEN) IOAPIC[0]: apic_id 8, version 32, address 0xfec00000, GSI 0-23 (XEN) ACPI: IOAPIC (id[0x09] address[0xfec8a000] gsi_base[24]) (XEN) IOAPIC[1]: apic_id 9, version 32, address 0xfec8a000, GSI 24-47 (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) (XEN) ACPI: IRQ0 used by override. (XEN) ACPI: IRQ2 used by override. (XEN) ACPI: IRQ9 used by override. (XEN) Enabling APIC mode: Phys. Using 2 I/O APICs (XEN) ACPI: HPET id: 0x8086a301 base: 0xfed00000 (XEN) PCI: MCFG configuration 0: base e0000000 segment 0000 buses 00 - ff (XEN) PCI: MCFG area at e0000000 reserved in E820 (XEN) PCI: Using MCFG for segment 0000 bus 00-ff (XEN) [VT-D]Host address width 40 (XEN) [VT-D]found ACPI_DMAR_DRHD: (XEN) [VT-D] dmaru->address = fbffe000 (XEN) [VT-D]drhd->address = fbffe000 iommu->reg = ffff82c00021d000 (XEN) [VT-D]cap = c90780106f0462 ecap = f020fe (XEN) [VT-D] IOAPIC: 0000:f0:1f.7 (XEN) [VT-D] IOAPIC: 0000:00:13.0 (XEN) [VT-D] flags: INCLUDE_ALL (XEN) [VT-D]found ACPI_DMAR_RMRR: (XEN) [VT-D] endpoint: 0000:00:1d.0 (XEN) [VT-D] endpoint: 0000:00:1d.1 (XEN) [VT-D] endpoint: 0000:00:1d.2 (XEN) [VT-D] endpoint: 0000:00:1d.7 (XEN) [VT-D] endpoint: 0000:00:1a.0 (XEN) [VT-D] endpoint: 0000:00:1a.1 (XEN) [VT-D] endpoint: 0000:00:1a.2 (XEN) [VT-D] endpoint: 0000:00:1a.7 (XEN) [VT-D]found ACPI_DMAR_RMRR: (XEN) [VT-D] endpoint: 0000:00:1d.0 (XEN) [VT-D] endpoint: 0000:00:1d.1 (XEN) [VT-D] endpoint: 0000:00:1d.2 (XEN) [VT-D] endpoint: 0000:00:1d.7 (XEN) [VT-D] endpoint: 0000:00:1a.0 (XEN) [VT-D] endpoint: 0000:00:1a.1 (XEN) [VT-D] endpoint: 0000:00:1a.2 (XEN) [VT-D] endpoint: 0000:00:1a.7 (XEN) [VT-D]found ACPI_DMAR_ATSR: (XEN) [VT-D] atsru->all_ports: 0 (XEN) [VT-D] bridge: 0000:00:01.0 start=0 sec=1 sub=1 (XEN) [VT-D] bridge: 0000:00:03.0 start=0 sec=2 sub=2 (XEN) [VT-D] bridge: 0000:00:05.0 start=0 sec=3 sub=3 (XEN) [VT-D] bridge: 0000:00:07.0 start=0 sec=4 sub=4 (XEN) [VT-D] bridge: 0000:00:09.0 start=0 sec=5 sub=5 (XEN) Failed to get Error Log Address Range. (XEN) HEST: Table parsing has been initialized (XEN) Using ACPI (MADT) for SMP configuration information (XEN) SMP: Allowing 24 CPUs (8 hotplug CPUs) (XEN) IRQ limits: 48 GSI, 3040 MSI/MSI-X (XEN) CPU0: Intel machine check reporting enabled (XEN) Speculative mitigation facilities: (XEN) Hardware features: (XEN) Compiled-in support: SHADOW_PAGING (XEN) Xen settings: BTI-Thunk N/A, SPEC_CTRL: No, Other: (XEN) L1TF: believed vulnerable, maxphysaddr L1D 44, CPUID 40, Safe address 10000000000 (XEN) Support for HVM VMs: RSB EAGER_FPU (XEN) Support for PV VMs: RSB EAGER_FPU (XEN) XPTI (64-bit PV only): Dom0 enabled, DomU enabled (without PCID) (XEN) PV L1TF shadowing: Dom0 disabled, DomU enabled (XEN) Using scheduler: SMP Credit Scheduler rev2 (credit2) (XEN) Initializing Credit2 scheduler (XEN) load_precision_shift: 18 (XEN) load_window_shift: 30 (XEN) underload_balance_tolerance: 0 (XEN) overload_balance_tolerance: -3 (XEN) runqueues arrangement: socket (XEN) cap enforcement granularity: 10ms (XEN) load tracking window length 1073741824 ns (XEN) Adding cpu 0 to runqueue 0 (XEN) First cpu on runqueue, activating (XEN) Platform timer is 14.318MHz HPET (XEN) Detected 2266.757 MHz processor. (XEN) Initing memory sharing. (XEN) alt table ffff82d0804309b8 -> ffff82d0804318d0 (XEN) Intel VT-d iommu 0 supported page sizes: 4kB. (XEN) Intel VT-d Snoop Control enabled. (XEN) Intel VT-d Dom0 DMA Passthrough not enabled. (XEN) Intel VT-d Queued Invalidation enabled. (XEN) Intel VT-d Interrupt Remapping enabled. (XEN) Intel VT-d Posted Interrupt not enabled. (XEN) Intel VT-d Shared EPT tables not enabled. (XEN) I/O virtualisation enabled (XEN) - Dom0 mode: Relaxed (XEN) Interrupt remapping enabled (XEN) nr_sockets: 3 (XEN) ENABLING IO-APIC IRQs (XEN) -> Using new ACK method (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1 (XEN) Allocated console ring of 128 KiB. (XEN) mwait-idle: MWAIT substates: 0x1120 (XEN) mwait-idle: v0.4.1 model 0x1a (XEN) mwait-idle: lapic_timer_reliable_states 0x2 (XEN) HPET: 0 timers usable for broadcast (4 total) (XEN) VMX: Supported advanced features: (XEN) - APIC MMIO access virtualisation (XEN) - APIC TPR shadow (XEN) - Extended Page Tables (EPT) (XEN) - Virtual-Processor Identifiers (VPID) (XEN) - Virtual NMI (XEN) - MSR direct-access bitmap (XEN) HVM: ASIDs enabled. (XEN) HVM: VMX enabled (XEN) HVM: Hardware Assisted Paging (HAP) detected (XEN) HVM: HAP page sizes: 4kB, 2MB (XEN) Adding cpu 1 to runqueue 0 (XEN) Adding cpu 2 to runqueue 0 (XEN) Adding cpu 3 to runqueue 0 (XEN) Adding cpu 4 to runqueue 0 (XEN) Adding cpu 5 to runqueue 0 (XEN) Adding cpu 6 to runqueue 0 (XEN) Adding cpu 7 to runqueue 0 (XEN) Adding cpu 8 to runqueue 1 (XEN) First cpu on runqueue, activating (XEN) Adding cpu 9 to runqueue 1 (XEN) Adding cpu 10 to runqueue 1 (XEN) Adding cpu 11 to runqueue 1 (XEN) Adding cpu 12 to runqueue 1 (XEN) Adding cpu 13 to runqueue 1 (XEN) Adding cpu 14 to runqueue 1 (XEN) Adding cpu 15 to runqueue 1 (XEN) Brought up 16 CPUs (XEN) build-id: aba2ec2f0b48776dac7a6e615962fec7240f6447 (XEN) ACPI sleep modes: S3 (XEN) VPMU: disabled (XEN) mcheck_poll: Machine check polling timer started. (XEN) Dom0 has maximum 816 PIRQs (XEN) NX (Execute Disable) protection active (XEN) *** Building a PVH Dom0 *** (XEN) [VT-D]d0:Hostbridge: skip 0000:00:00.0 map (XEN) Masked UR signaling on 0000:00:00.0 (XEN) Masked UR signaling on 0000:00:01.0 (XEN) Masked UR signaling on 0000:00:03.0 (XEN) Masked UR signaling on 0000:00:05.0 (XEN) Masked UR signaling on 0000:00:07.0 (XEN) Masked UR signaling on 0000:00:09.0 (XEN) [VT-D]d0:PCI: map 0000:00:13.0 (XEN) [VT-D]d0:PCIe: map 0000:00:14.0 (XEN) Masked VT-d error signaling on 0000:00:14.0 (XEN) [VT-D]d0:PCIe: map 0000:00:14.1 (XEN) [VT-D]d0:PCIe: map 0000:00:14.2 (XEN) [VT-D]d0:PCI: map 0000:00:14.3 (XEN) [VT-D]d0:PCIe: map 0000:00:16.0 (XEN) [VT-D]d0:PCIe: map 0000:00:16.1 (XEN) [VT-D]d0:PCIe: map 0000:00:16.2 (XEN) [VT-D]d0:PCIe: map 0000:00:16.3 (XEN) [VT-D]d0:PCIe: map 0000:00:16.4 (XEN) [VT-D]d0:PCIe: map 0000:00:16.5 (XEN) [VT-D]d0:PCIe: map 0000:00:16.6 (XEN) [VT-D]d0:PCIe: map 0000:00:16.7 (XEN) [VT-D]d0:PCI: map 0000:00:1a.0 (XEN) [VT-D]d0:PCI: map 0000:00:1a.1 (XEN) [VT-D]d0:PCI: map 0000:00:1a.2 (XEN) [VT-D]d0:PCI: map 0000:00:1a.7 (XEN) [VT-D]d0:PCI: map 0000:00:1d.0 (XEN) [VT-D]d0:PCI: map 0000:00:1d.1 (XEN) [VT-D]d0:PCI: map 0000:00:1d.2 (XEN) [VT-D]d0:PCI: map 0000:00:1d.7 (XEN) [VT-D]d0:PCI: map 0000:00:1f.0 (XEN) [VT-D]d0:PCI: map 0000:00:1f.2 (XEN) [VT-D]d0:PCI: map 0000:00:1f.3 (XEN) [VT-D]d0:PCI: map 0000:00:1f.5 (XEN) [VT-D]d0:PCIe: map 0000:01:00.0 (XEN) [VT-D]d0:PCIe: map 0000:01:00.1 (XEN) [VT-D]d0:PCIe: map 0000:03:00.0 (XEN) [VT-D]d0:PCIe: map 0000:03:00.1 (XEN) [VT-D]d0:PCI: map 0000:06:01.0 (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:00.0 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:00.1 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:02.0 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:02.1 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:02.4 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:02.5 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:03.0 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:03.1 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:03.2 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:03.4 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:04.0 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:04.1 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:04.2 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:04.3 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:05.0 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:05.1 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:05.2 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:05.3 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:06.0 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:06.1 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:06.2 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:06.3 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:00.0 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:00.1 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:02.0 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:02.1 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:02.4 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:02.5 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:03.0 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:03.1 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:03.2 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:03.4 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:04.0 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:04.1 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:04.2 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:04.3 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:05.0 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:05.1 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:05.2 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:05.3 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:06.0 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:06.1 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:06.2 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:06.3 map (XEN) [VT-D]iommu_enable_translation: iommu->reg = ffff82c00021d000 (XEN) Dom0 memory allocation stats: (XEN) order 0 allocations: 2 (XEN) order 1 allocations: 1 (XEN) order 2 allocations: 1 (XEN) order 3 allocations: 1 (XEN) order 4 allocations: 1 (XEN) order 5 allocations: 1 (XEN) order 6 allocations: 1 (XEN) order 7 allocations: 3 (XEN) order 8 allocations: 2 (XEN) order 9 allocations: 2 (XEN) order 10 allocations: 2 (XEN) order 11 allocations: 2 (XEN) order 12 allocations: 2 (XEN) order 13 allocations: 2 (XEN) order 14 allocations: 2 (XEN) order 15 allocations: 2 (XEN) order 16 allocations: 2 (XEN) order 17 allocations: 2 (XEN) order 18 allocations: 2 (XEN) WARNING: PVH is an experimental mode with limited functionality (XEN) Initial low memory virq threshold set at 0x4000 pages. (XEN) Scrubbing Free RAM in background (XEN) Std. Loglevel: All (XEN) Guest Loglevel: All (XEN) *************************************************** (XEN) WARNING: CONSOLE OUTPUT IS SYNCHRONOUS (XEN) This option is intended to aid debugging of Xen by ensuring (XEN) that all output is synchronously delivered on the serial line. (XEN) However it can introduce SIGNIFICANT latencies and affect (XEN) timekeeping. It is NOT recommended for production use! (XEN) *************************************************** (XEN) Booted on L1TF-vulnerable hardware with SMT/Hyperthreading (XEN) enabled. Please assess your configuration and choose an (XEN) explicit 'smt=' setting. See XSA-273. (XEN) *************************************************** (XEN) Booted on MLPDS/MFBDS-vulnerable hardware with SMT/Hyperthreading (XEN) enabled. Mitigations will not be fully effective. Please (XEN) choose an explicit smt= setting. See XSA-297. (XEN) *************************************************** (XEN) 3... 2... 1... (XEN) Xen is relinquishing VGA console. (XEN) *** Serial input to DOM0 (type 'CTRL-a' three times to switch input) (XEN) Freed 500kB init memory ---<>--- Copyright (c) 1992-2018 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 12.0-RELEASE-p10 GENERIC amd64 FreeBSD clang version 6.0.1 (tags/RELEASE_601/final 335540) (based on LLVM 6.0.1) VT(vga): text 80x25 XEN: Hypervisor version 4.12 detected. CPU: Intel(R) Xeon(R) CPU E5520 @ 2.27GHz (2266.76-MHz K8-class CPU) Origin="GenuineIntel" Id=0x106a5 Family=0x6 Model=0x1a Stepping=5 Features=0x1fc3fbff Features2=0x80982201 AMD Features=0x28100800 AMD Features2=0x1 TSC: P-state invariant Hypervisor: Origin = "XenVMMXenVMM" real memory = 5378043904 (5128 MB) avail memory = 4099301376 (3909 MB) Event timer "LAPIC" quality 100 ACPI APIC Table: <080312 APIC1521> FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs FreeBSD/SMP: 1 package(s) x 4 core(s) random: unblocking device. Firmware Warning (ACPI): 32/64X length mismatch in FADT/Gpe0Block: 128/64 (20181003/tbfadt-748) ioapic1: Changing APIC ID to 9 ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-24 on motherboard Launching APs: 2 1 3 random: entropy device external interface kbd1 at kbdmux0 netmap: loaded module [ath_hal] loaded module_register_init: MOD_LOAD (vesa, 0xffffffff810f9a00, 0) error 19 nexus0 vtvga0: on motherboard cryptosoft0: on motherboard acpi0: on motherboard (XEN) d0: bind: m_gsi=9 g_gsi=9 acpi0: Power Button (fixed) cpu0: on acpi0 attimer0: port 0x40-0x43 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 atrtc0: port 0x70-0x71 irq 8 on acpi0 atrtc0: registered as a time-of-day clock, resolution 1.000000s (XEN) d0: bind: m_gsi=8 g_gsi=8 Event timer "RTC" frequency 32768 Hz quality 0 hpet0: iomem 0xfed03000-0xfed033ff on acpi0 hpet0: HPET never increments, disabling device_attach: hpet0 attach returned 6 Timecounter "ACPI-safe" frequency 3579545 Hz quality 850 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: at device 1.0 on pci0 pci1: on pcib1 igb0: port 0xdc00-0xdc1f mem 0xfbce0000-0xfbcfffff,0xfbcc0000-0xfbcdffff,0xfbc9c000-0xfbc9ffff irq 28 at device 0.0 on pci1 igb0: attach_pre capping queues at 8 igb0: using 1024 tx descriptors and 1024 rx descriptors igb0: msix_init qsets capped at 8 igb0: pxm cpus: 4 queue msgs: 9 admincnt: 1 igb0: using 4 rx queues 4 tx queues igb0: Using MSIX interrupts with 5 vectors igb0: allocated for 4 tx_queues igb0: allocated for 4 rx_queues igb0: Ethernet address: 00:25:90:36:fd:2e igb0: netmap queues/slots: TX 4/1024, RX 4/1024 igb1: port 0xd880-0xd89f mem 0xfbc60000-0xfbc7ffff,0xfbc40000-0xfbc5ffff,0xfbc1c000-0xfbc1ffff irq 40 at device 0.1 on pci1 igb1: attach_pre capping queues at 8 igb1: using 1024 tx descriptors and 1024 rx descriptors igb1: msix_init qsets capped at 8 igb1: pxm cpus: 4 queue msgs: 9 admincnt: 1 igb1: using 4 rx queues 4 tx queues igb1: Using MSIX interrupts with 5 vectors igb1: allocated for 4 tx_queues igb1: allocated for 4 rx_queues igb1: Ethernet address: 00:25:90:36:fd:2f igb1: netmap queues/slots: TX 4/1024, RX 4/1024 pcib2: at device 3.0 on pci0 pci2: on pcib2 pcib3: at device 5.0 on pci0 pci3: on pcib3 igb2: port 0xec00-0xec1f mem 0xfbde0000-0xfbdfffff,0xfbdc0000-0xfbddffff,0xfbd9c000-0xfbd9ffff irq 26 at device 0.0 on pci3 igb2: attach_pre capping queues at 8 igb2: using 1024 tx descriptors and 1024 rx descriptors igb2: msix_init qsets capped at 8 igb2: pxm cpus: 4 queue msgs: 9 admincnt: 1 igb2: using 4 rx queues 4 tx queues igb2: Using MSIX interrupts with 5 vectors igb2: allocated for 4 tx_queues igb2: allocated for 4 rx_queues igb2: Ethernet address: 00:25:90:09:c8:da igb2: netmap queues/slots: TX 4/1024, RX 4/1024 igb3: port 0xe880-0xe89f mem 0xfbd60000-0xfbd7ffff,0xfbd40000-0xfbd5ffff,0xfbd1c000-0xfbd1ffff irq 25 at device 0.1 on pci3 igb3: attach_pre capping queues at 8 igb3: using 1024 tx descriptors and 1024 rx descriptors igb3: msix_init qsets capped at 8 igb3: pxm cpus: 4 queue msgs: 9 admincnt: 1 igb3: using 4 rx queues 4 tx queues igb3: Using MSIX interrupts with 5 vectors igb3: allocated for 4 tx_queues igb3: allocated for 4 rx_queues igb3: Ethernet address: 00:25:90:09:c8:db igb3: netmap queues/slots: TX 4/1024, RX 4/1024 pcib4: at device 7.0 on pci0 pci4: on pcib4 pcib5: at device 9.0 on pci0 pci5: on pcib5 pci0: at device 20.0 (no driver attached) pci0: at device 20.1 (no driver attached) pci0: at device 20.2 (no driver attached) pci0: at device 20.3 (no driver attached) uhci0: port 0xcc00-0xcc1f irq 16 at device 26.0 on pci0 (XEN) d0: bind: m_gsi=16 g_gsi=16 uhci0: LegSup = 0x2f00 usbus0 on uhci0 usbus0: 12Mbps Full Speed USB v1.0 uhci1: port 0xc880-0xc89f irq 21 at device 26.1 on pci0 (XEN) d0: bind: m_gsi=21 g_gsi=21 uhci1: LegSup = 0x2f00 usbus1 on uhci1 usbus1: 12Mbps Full Speed USB v1.0 uhci2: port 0xc800-0xc81f irq 19 at device 26.2 on pci0 (XEN) d0: bind: m_gsi=19 g_gsi=19 uhci2: LegSup = 0x2f00 usbus2 on uhci2 usbus2: 12Mbps Full Speed USB v1.0 ehci0: mem 0xfbeda000-0xfbeda3ff irq 18 at device 26.7 on pci0 (XEN) d0: bind: m_gsi=18 g_gsi=18 usbus3: EHCI version 1.0 usbus3 on ehci0 usbus3: 480Mbps High Speed USB v2.0 uhci3: port 0xc480-0xc49f irq 23 at device 29.0 on pci0 (XEN) d0: bind: m_gsi=23 g_gsi=23 uhci3: LegSup = 0x2f00 usbus4 on uhci3 usbus4: 12Mbps Full Speed USB v1.0 uhci4: port 0xc400-0xc41f irq 19 at device 29.1 on pci0 uhci4: LegSup = 0x2f00 usbus5 on uhci4 usbus5: 12Mbps Full Speed USB v1.0 uhci5: port 0xc080-0xc09f irq 18 at device 29.2 on pci0 uhci5: LegSup = 0x2f00 usbus6 on uhci5 usbus6: 12Mbps Full Speed USB v1.0 ehci1: mem 0xfbed8000-0xfbed83ff irq 23 at device 29.7 on pci0 usbus7: EHCI version 1.0 usbus7 on ehci1 usbus7: 480Mbps High Speed USB v2.0 pcib6: at device 30.0 on pci0 pci6: on pcib6 vgapci0: mem 0xf9000000-0xf9ffffff,0xfaffc000-0xfaffffff,0xfb000000-0xfb7fffff irq 18 at device 1.0 on pci6 vgapci0: Boot video device isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0xc000-0xc007,0xbc00-0xbc03,0xb880-0xb887,0xb800-0xb803,0xb480-0xb48f,0xb400-0xb40f irq 19 at device 31.2 on pci0 ata2: at channel 0 on atapci0 ata3: at channel 1 on atapci0 atapci1: port 0xb000-0xb007,0xac00-0xac03,0xa880-0xa887,0xa800-0xa803,0xa480-0xa48f,0xa400-0xa40f irq 19 at device 31.5 on pci0 ata4: at channel 0 on atapci1 ata5: at channel 1 on atapci1 acpi_button0: on acpi0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 (XEN) d0: bind: m_gsi=4 g_gsi=4 xenpv0: on motherboard granttable0: on xenpv0 xen_et0: on xenpv0 Event timer "XENTIMER" frequency 1000000000 Hz quality 950 Timecounter "XENTIMER" frequency 1000000000 Hz quality 950 xen_et0: registered as a time-of-day clock, resolution 0.000001s xenstore0: on xenpv0 xsd_dev0: on xenpv0 evtchn0: on xenpv0 privcmd0: on xenpv0 gntdev0: on xenpv0 debug0: on xenpv0 xc0: on xenpv0 ipmi0: on isa0 ipmi0: KCS mode found at io 0xca2 alignment 0x1 on isa orm0: at iomem 0xc0000-0xc7fff,0xc8000-0xc8fff,0xc9000-0xc9fff,0xca000-0xcafff pnpid ORM0000 on isa0 vga0: at port 0x3d0-0x3db iomem 0xb8000-0xbffff pnpid PNP0900 on isa0 atkbdc0: at port 0x60,0x64 on isa0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 (XEN) d0: bind: m_gsi=1 g_gsi=1 atkbd0: [GIANT-LOCKED] atkbdc0: non-PNP ISA device will be removed from GENERIC in FreeBSD 12. Timecounters tick every 10.000 msec ipmi0: IPMI device rev. 1, firmware rev. 3.13, version 2.0, device support mask 0xbf ¿€¹ipmi0: Attached watchdog ipmi0: Establishing power cycle handler ugen7.1: at usbus7 ugen5.1: at usbus5 uhub0: on usbus7 ugen1.1: at usbus1 ugen3.1: at usbus3 ugen4.1: at usbus4 uhub1: on usbus5 uhub2: on usbus4 uhub3: on usbus1 ugen2.1: at usbus2 uhub4: on usbus3 ugen0.1: at usbus0 uhub5: on usbus2 uhub6: on usbus0 ugen6.1: at usbus6 uhub7: on usbus6 uhub1: 2 ports with 2 removable, self powered uhub2: 2 ports with 2 removable, self powered uhub3: 2 ports with 2 removable, self powered uhub5: 2 ports with 2 removable, self powered uhub6: 2 ports with 2 removable, self powered uhub7: 2 ports with 2 removable, self powered ada0 at ata2 bus 0 scbus0 target 0 lun 0 ada0: ACS-3 ATA SATA 3.x device ada0: Serial Number ZGY0KSAS ada0: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes) ada0: 3815447MB (7814037168 512 byte sectors) ada0: quirks=0x1<4K> ada1 at ata2 bus 0 scbus0 target 1 lun 0 ada1: ACS-3 ATA SATA 3.x device ada1: Serial Number ZGY0KS9B ada1: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes) ada1: 3815447MB (7814037168 512 byte sectors) ada1: quirks=0x1<4K> ada2 at ata3 bus 0 scbus1 target 0 lun 0 ada2: ACS-3 ATA SATA 3.x device ada2: Serial Number ZGY0KSC9 ada2: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes) ada2: 3815447MB (7814037168 512 byte sectors) ada2: quirks=0x1<4K> ada3 at ata3 bus 0 scbus1 target 1 lun 0 ada3: ACS-3 ATA SATA 3.x device ada3: Serial Number ZDH1SGEB ada3: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes) ada3: 3815447MB (7814037168 512 byte sectors) ada3: quirks=0x1<4K> Trying to mount root from ufs:/dev/gpt/gptroot [rw]... uhub0: 6 ports with 6 removable, self powered uhub4: 6 ports with 6 removable, self powered cd0 at ata4 bus 0 scbus2 target 0 lun 0 cd0: Removable CD-ROM SCSI device cd0: 150.000MB/s transfers (SATA 1.x, UDMA2, ATAPI 12bytes, PIO 8192bytes) cd0: Attempt to query device size failed: NOT READY, Medium not present - tray closed GEOM_MIRROR: Device mirror/gm0 launched (1/2). GEOM_MIRROR: Device gm0: rebuilding provider ada1. usb_alloc_device: set address 2 failed (USB_ERR_TIMEOUT, ignored) usbd_setup_device_desc: getting device descriptor at addr 2 failed, USB_ERR_TIMEOUT usbd_req_re_enumerate: addr=2, set address failed! (USB_ERR_TIMEOUT, ignored) usbd_setup_device_desc: getting device descriptor at addr 2 failed, USB_ERR_TIMEOUT usbd_req_re_enumerate: addr=2, set address failed! (USB_ERR_TIMEOUT, ignored) usbd_setup_device_desc: getting device descriptor at addr 2 failed, USB_ERR_TIMEOUT usbd_req_re_enumerate: addr=2, set address failed! (USB_ERR_TIMEOUT, ignored) usbd_setup_device_desc: getting device descriptor at addr 2 failed, USB_ERR_TIMEOUT usbd_req_re_enumerate: addr=2, set address failed! (USB_ERR_TIMEOUT, ignored) usbd_setup_device_desc: getting device descriptor at addr 2 failed, USB_ERR_TIMEOUT ugen1.2: at usbus1 (disconnected) uhub_reattach_port: could not allocate new device WARNING: /mnt was not properly dismounted load: 0.08 cmd: sh 29 [biord] 274.71r 0.00u 0.00s 0% 764k From owner-freebsd-xen@freebsd.org Fri Oct 11 21:53:30 2019 Return-Path: Delivered-To: freebsd-xen@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A845113374B for ; Fri, 11 Oct 2019 21:53:30 +0000 (UTC) (envelope-from buhrow@lothlorien.nfbcal.org) Received: from lothlorien.nfbcal.org (ns.NFBCAL.ORG [157.22.230.125]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "nfbcal.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46qhXn35V7z44qv for ; Fri, 11 Oct 2019 21:53:29 +0000 (UTC) (envelope-from buhrow@lothlorien.nfbcal.org) Received: from lothlorien.nfbcal.org (localhost [127.0.0.1]) by lothlorien.nfbcal.org (8.15.2/8.14.1-NFBNETBSD) with ESMTPS id x9BLrQwd009994 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 11 Oct 2019 14:53:26 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.97.3 at lothlorien.nfbcal.org Received: (from buhrow@localhost) by lothlorien.nfbcal.org (8.15.2/8.12.11) id x9BLrQSI028763; Fri, 11 Oct 2019 14:53:26 -0700 (PDT) Message-Id: <201910112153.x9BLrQSI028763@lothlorien.nfbcal.org> From: Brian Buhrow Date: Fri, 11 Oct 2019 14:53:26 -0700 In-Reply-To: <201910102323.x9ANNACe003999@lothlorien.nfbcal.org> X-Mailer: Mail User's Shell (7.2.6 beta(4.pl1)+dynamic 20000103) To: freebsd-xen@freebsd.org Subject: Re: i/o is very slow on FreeBSD dom0 with Xen-4.12 and Freebsd-12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (lothlorien.nfbcal.org [127.0.0.1]); Fri, 11 Oct 2019 14:53:26 -0700 (PDT) X-Rspamd-Queue-Id: 46qhXn35V7z44qv X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of buhrow@lothlorien.nfbcal.org designates 157.22.230.125 as permitted sender) smtp.mailfrom=buhrow@lothlorien.nfbcal.org X-Spamd-Result: default: False [-2.00 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.993,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+a:ns.nfbcal.org:c]; NEURAL_HAM_LONG(-1.00)[-0.999,0]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; DMARC_NA(0.00)[nfbcal.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; IP_SCORE(-0.01)[country: US(-0.05)]; FORGED_SENDER(0.30)[buhrow@nfbcal.org,buhrow@lothlorien.nfbcal.org]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:7091, ipnet:157.22.0.0/16, country:US]; FROM_NEQ_ENVFROM(0.00)[buhrow@nfbcal.org,buhrow@lothlorien.nfbcal.org]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Oct 2019 21:53:30 -0000 hello. Another data point on this issue. It appears the problem may be with the onboard IDE controllers, or, possibly, the way interrupts are handled. If I switch the BIOS to enable ahcisata mode instead of IDE mode for the SATA controllers, things boot more quickly, but the FreeBSD kernel is unable to talk to the hard drives. The ahci channels time out and we get stuck in an endless loop of CAM system retries. When the BIOS is set to Enhanced IDE mode, the FreeBSD kernel can talk to the drives, but the data rate is so slow as to be unusable. When FreeBSD boots on bare metal it can talk to the ahci or IDE drives without a problem. This makes me think Xen is messing with the interrupt handling of the IDE controller somehow. Is it possible the BIOS sets things up in a way that FreeBSD likes, but Xen does not? Perhaps the ioapic is programmed in a way that FreeBSD doesn't like? -thanks -Brian From owner-freebsd-xen@freebsd.org Fri Oct 11 23:22:51 2019 Return-Path: Delivered-To: freebsd-xen@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D2D64135128 for ; Fri, 11 Oct 2019 23:22:51 +0000 (UTC) (envelope-from buhrow@lothlorien.nfbcal.org) Received: from lothlorien.nfbcal.org (ns.NFBCAL.ORG [157.22.230.125]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "nfbcal.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46qkWt2fXFz47r0 for ; Fri, 11 Oct 2019 23:22:49 +0000 (UTC) (envelope-from buhrow@lothlorien.nfbcal.org) Received: from lothlorien.nfbcal.org (localhost [127.0.0.1]) by lothlorien.nfbcal.org (8.15.2/8.14.1-NFBNETBSD) with ESMTPS id x9BNMZ7R018406 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 11 Oct 2019 16:22:35 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.97.3 at lothlorien.nfbcal.org Received: (from buhrow@localhost) by lothlorien.nfbcal.org (8.15.2/8.12.11) id x9BNMWlp020596; Fri, 11 Oct 2019 16:22:32 -0700 (PDT) Message-Id: <201910112322.x9BNMWlp020596@lothlorien.nfbcal.org> From: Brian Buhrow Date: Fri, 11 Oct 2019 16:22:32 -0700 In-Reply-To: <201910112153.x9BLrQSI028763@lothlorien.nfbcal.org> X-Mailer: Mail User's Shell (7.2.6 beta(4.pl1)+dynamic 20000103) To: freebsd-xen@freebsd.org Subject: Re: i/o is very slow on FreeBSD dom0 with Xen-4.12 and Freebsd-12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (lothlorien.nfbcal.org [127.0.0.1]); Fri, 11 Oct 2019 16:22:36 -0700 (PDT) X-Rspamd-Queue-Id: 46qkWt2fXFz47r0 X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of buhrow@lothlorien.nfbcal.org designates 157.22.230.125 as permitted sender) smtp.mailfrom=buhrow@lothlorien.nfbcal.org X-Spamd-Result: default: False [-2.00 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.992,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+a:ns.nfbcal.org]; NEURAL_HAM_LONG(-1.00)[-0.997,0]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; DMARC_NA(0.00)[nfbcal.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; IP_SCORE(-0.01)[country: US(-0.05)]; FORGED_SENDER(0.30)[buhrow@nfbcal.org,buhrow@lothlorien.nfbcal.org]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:7091, ipnet:157.22.0.0/16, country:US]; FROM_NEQ_ENVFROM(0.00)[buhrow@nfbcal.org,buhrow@lothlorien.nfbcal.org]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Oct 2019 23:22:51 -0000 hello. Following up on this issue yet again, I'm pretty sure I have the hardware discussed in the link below. I tried disabling interrupt remapping as the article suggests, but that just makes matters worse. As I understand it, when FreeBSD is running as a dom0, it must have iommu interrupt remapping enabled in order to function. Should I conclude that this hardware just cannot run FreeBSD as a dom0 or is there another work around that I might use? -thanks -Brian https://support.citrix.com/article/CTX136517 From owner-freebsd-xen@freebsd.org Sat Oct 12 14:57:56 2019 Return-Path: Delivered-To: freebsd-xen@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id DD4C613731C for ; Sat, 12 Oct 2019 14:57:56 +0000 (UTC) (envelope-from buhrow@lothlorien.nfbcal.org) Received: from lothlorien.nfbcal.org (ns.NFBCAL.ORG [157.22.230.125]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "nfbcal.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46r7Gp1691z49nc for ; Sat, 12 Oct 2019 14:57:53 +0000 (UTC) (envelope-from buhrow@lothlorien.nfbcal.org) Received: from lothlorien.nfbcal.org (localhost [127.0.0.1]) by lothlorien.nfbcal.org (8.15.2/8.14.1-NFBNETBSD) with ESMTPS id x9CEvjtQ018346 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sat, 12 Oct 2019 07:57:45 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.97.3 at lothlorien.nfbcal.org Received: (from buhrow@localhost) by lothlorien.nfbcal.org (8.15.2/8.12.11) id x9CEviM4026245; Sat, 12 Oct 2019 07:57:44 -0700 (PDT) Message-Id: <201910121457.x9CEviM4026245@lothlorien.nfbcal.org> From: Brian Buhrow Date: Sat, 12 Oct 2019 07:57:44 -0700 In-Reply-To: <20191012095900.GA14005@Air-de-Roger> X-Mailer: Mail User's Shell (7.2.6 beta(4.pl1)+dynamic 20000103) To: Roger Pau =?iso-8859-1?Q?Monn=E9?= Subject: Re: i/o is very slow on FreeBSD dom0 with Xen-4.12 and Freebsd-12 Cc: , buhrow@nfbcal.org X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (lothlorien.nfbcal.org [127.0.0.1]); Sat, 12 Oct 2019 07:57:45 -0700 (PDT) X-Rspamd-Queue-Id: 46r7Gp1691z49nc X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of buhrow@lothlorien.nfbcal.org designates 157.22.230.125 as permitted sender) smtp.mailfrom=buhrow@lothlorien.nfbcal.org X-Spamd-Result: default: False [-2.13 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.994,0]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+a:ns.nfbcal.org]; FROM_HAS_DN(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[nfbcal.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; IP_SCORE(-0.14)[ipnet: 157.22.0.0/16(-0.36), asn: 7091(-0.29), country: US(-0.05)]; FORGED_SENDER(0.30)[buhrow@nfbcal.org,buhrow@lothlorien.nfbcal.org]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:7091, ipnet:157.22.0.0/16, country:US]; FROM_NEQ_ENVFROM(0.00)[buhrow@nfbcal.org,buhrow@lothlorien.nfbcal.org]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Oct 2019 14:57:56 -0000 hello. Here is the log with no-intremap enabled. And, thank you for the help. I hope your PTO was refreshing. -Brian boot_serial="YES" # -h: Use serial console comconsole_speed="115200" # Set the current serial console speed comconsole_port="0x2f8" # Set the current serial console port #console="vidconsole" # A comma separated list of console(s) console="comconsole" # A comma separated list of console(s) geom_mirror_load="YES" # RAID1 disk driver (see gmirror(8)) ahci_load="YES" # ahci driver ipmi_load="YES" # Ripmi driver if_tap_load="YES" # Load bridge driver. if_vlan_load="YES" # Load vlan driver. # Turn on Xen (BB 10/08/2019) xen_kernel="/boot/xen" xen_cmdline="dom0_mem=4096m dom0_max_vcpus=4 dom0=pvh,verbose console=com2,vga iommu=no-intremap,verbose,debug sync_console=true com2=115200,8n1 guest_loglvl=all loglvl=all" (XEN) Xen version 4.12.1 (buhrow@) (FreeBSD clang version 6.0.1 (tags/RELEASE_601/final 335540) (based on LLVM 6.0.1)) =n Thu Oct 3 11:34:20 PDT 2019 (XEN) Latest ChangeSet: (XEN) Console output is synchronous. (XEN) Bootloader: FreeBSD Loader (XEN) Command line: dom0_mem=4096m dom0_max_vcpus=4 dom0=pvh,verbose console=com2,vga iommu=no-intremap,verbose, sync_console=true com2=115200,8n1 guest_loglvl=all loglvl=all (XEN) Xen image load base address: 0 (XEN) Video information: (XEN) VGA is text mode 80x25, font 8x16 (XEN) VBE/DDC methods: none; EDID transfer time: 0 seconds (XEN) EDID info not retrieved because no DDC retrieval method detected (XEN) Disc information: (XEN) Found 4 MBR signatures (XEN) Found 4 EDD information structures (XEN) Xen-e820 RAM map: (XEN) 0000000000000000 - 0000000000098400 (usable) (XEN) 0000000000098400 - 00000000000a0000 (reserved) (XEN) 00000000000e0000 - 0000000000100000 (reserved) (XEN) 0000000000100000 - 00000000bf780000 (usable) (XEN) 00000000bf78e000 - 00000000bf790000 type 9 (XEN) 00000000bf790000 - 00000000bf79e000 (ACPI data) (XEN) 00000000bf79e000 - 00000000bf7d0000 (ACPI NVS) (XEN) 00000000bf7d0000 - 00000000bf7e0000 (reserved) (XEN) 00000000bf7ec000 - 00000000c0000000 (reserved) (XEN) 00000000e0000000 - 00000000f0000000 (reserved) (XEN) 00000000fee00000 - 00000000fee01000 (reserved) (XEN) 00000000ffc00000 - 0000000100000000 (reserved) (XEN) 0000000100000000 - 0000001840000000 (usable) (XEN) New Xen image base address: 0xbf000000 (XEN) ACPI: RSDP 000FACE0, 0024 (r2 ACPIAM) (XEN) ACPI: XSDT BF790100, 008C (r1 SMCI 20120803 MSFT 97) (XEN) ACPI: FACP BF790290, 00F4 (r3 080312 FACP1521 20120803 MSFT 97) (XEN) ACPI: DSDT BF7906A0, 6580 (r1 10600 10600000 0 INTL 20051117) (XEN) ACPI: FACS BF79E000, 0040 (XEN) ACPI: APIC BF790390, 011E (r1 080312 APIC1521 20120803 MSFT 97) (XEN) ACPI: MCFG BF7904B0, 003C (r1 080312 OEMMCFG 20120803 MSFT 97) (XEN) ACPI: SLIT BF7904F0, 0030 (r1 080312 OEMSLIT 20120803 MSFT 97) (XEN) ACPI: OEMB BF79E040, 0086 (r1 080312 OEMB1521 20120803 MSFT 97) (XEN) ACPI: SRAT BF79A6A0, 01D0 (r1 080312 OEMSRAT 1 INTL 1) (XEN) ACPI: HPET BF79A870, 0038 (r1 080312 OEMHPET 20120803 MSFT 97) (XEN) ACPI: DMAR BF79E0D0, 0130 (r1 AMI OEMDMAR 1 MSFT 97) (XEN) ACPI: SSDT BF7A1710, 0363 (r1 DpgPmm CpuPm 12 INTL 20051117) (XEN) ACPI: EINJ BF79A8B0, 0130 (r1 AMIER AMI_EINJ 20120803 MSFT 97) (XEN) ACPI: BERT BF79AA40, 0030 (r1 AMIER AMI_BERT 20120803 MSFT 97) (XEN) ACPI: ERST BF79AA70, 01B0 (r1 AMIER AMI_ERST 20120803 MSFT 97) (XEN) ACPI: HEST BF79AC20, 00A8 (r1 AMIER ABC_HEST 20120803 MSFT 97) (XEN) System RAM: 98295MB (100654176kB) (XEN) SRAT: PXM 0 -> APIC 00 -> Node 0 (XEN) SRAT: PXM 0 -> APIC 02 -> Node 0 (XEN) SRAT: PXM 0 -> APIC 04 -> Node 0 (XEN) SRAT: PXM 0 -> APIC 06 -> Node 0 (XEN) SRAT: PXM 0 -> APIC 01 -> Node 0 (XEN) SRAT: PXM 0 -> APIC 03 -> Node 0 (XEN) SRAT: PXM 0 -> APIC 05 -> Node 0 (XEN) SRAT: PXM 0 -> APIC 07 -> Node 0 (XEN) SRAT: PXM 1 -> APIC 10 -> Node 1 (XEN) SRAT: PXM 1 -> APIC 12 -> Node 1 (XEN) SRAT: PXM 1 -> APIC 14 -> Node 1 (XEN) SRAT: PXM 1 -> APIC 16 -> Node 1 (XEN) SRAT: PXM 1 -> APIC 11 -> Node 1 (XEN) SRAT: PXM 1 -> APIC 13 -> Node 1 (XEN) SRAT: PXM 1 -> APIC 15 -> Node 1 (XEN) SRAT: PXM 1 -> APIC 17 -> Node 1 (XEN) SRAT: Node 0 PXM 0 0-a0000 (XEN) SRAT: Node 0 PXM 0 100000-c0000000 (XEN) SRAT: Node 0 PXM 0 100000000-c40000000 (XEN) SRAT: Node 1 PXM 1 c40000000-1840000000 (XEN) NUMA: Allocated memnodemap from 183e1b7000 - 183e1d0000 (XEN) NUMA: Using 8 for the hash shift. (XEN) Domain heap initialised DMA width 32 bits (XEN) CPU Vendor: Intel, Family 6 (0x6), Model 26 (0x1a), Stepping 5 (raw 000106a5) (XEN) found SMP MP-table at 000ff780 (XEN) DMI present. (XEN) Using APIC driver default (XEN) ACPI: PM-Timer IO Port: 0x808 (32 bits) (XEN) ACPI: SLEEP INFO: pm1x_cnt[1:804,1:0], pm1x_evt[1:800,1:0] (XEN) ACPI: wakeup_vec[bf79e00c], vec_size[20] (XEN) ACPI: Local APIC address 0xfee00000 (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled) (XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x10] enabled) (XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x02] enabled) (XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x12] enabled) (XEN) ACPI: LAPIC (acpi_id[0x05] lapic_id[0x04] enabled) (XEN) ACPI: LAPIC (acpi_id[0x06] lapic_id[0x14] enabled) (XEN) ACPI: LAPIC (acpi_id[0x07] lapic_id[0x06] enabled) (XEN) ACPI: LAPIC (acpi_id[0x08] lapic_id[0x16] enabled) (XEN) ACPI: LAPIC (acpi_id[0x09] lapic_id[0x01] enabled) (XEN) ACPI: LAPIC (acpi_id[0x0a] lapic_id[0x11] enabled) (XEN) ACPI: LAPIC (acpi_id[0x0b] lapic_id[0x03] enabled) (XEN) ACPI: LAPIC (acpi_id[0x0c] lapic_id[0x13] enabled) (XEN) ACPI: LAPIC (acpi_id[0x0d] lapic_id[0x05] enabled) (XEN) ACPI: LAPIC (acpi_id[0x0e] lapic_id[0x15] enabled) (XEN) ACPI: LAPIC (acpi_id[0x0f] lapic_id[0x07] enabled) (XEN) ACPI: LAPIC (acpi_id[0x10] lapic_id[0x17] enabled) (XEN) ACPI: LAPIC (acpi_id[0x11] lapic_id[0x90] disabled) (XEN) ACPI: LAPIC (acpi_id[0x12] lapic_id[0x91] disabled) (XEN) ACPI: LAPIC (acpi_id[0x13] lapic_id[0x92] disabled) (XEN) ACPI: LAPIC (acpi_id[0x14] lapic_id[0x93] disabled) (XEN) ACPI: LAPIC (acpi_id[0x15] lapic_id[0x94] disabled) (XEN) ACPI: LAPIC (acpi_id[0x16] lapic_id[0x95] disabled) (XEN) ACPI: LAPIC (acpi_id[0x17] lapic_id[0x96] disabled) (XEN) ACPI: LAPIC (acpi_id[0x18] lapic_id[0x97] disabled) (XEN) ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) (XEN) Overriding APIC driver with bigsmp (XEN) ACPI: IOAPIC (id[0x08] address[0xfec00000] gsi_base[0]) (XEN) IOAPIC[0]: apic_id 8, version 32, address 0xfec00000, GSI 0-23 (XEN) ACPI: IOAPIC (id[0x09] address[0xfec8a000] gsi_base[24]) (XEN) IOAPIC[1]: apic_id 9, version 32, address 0xfec8a000, GSI 24-47 (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) (XEN) ACPI: IRQ0 used by override. (XEN) ACPI: IRQ2 used by override. (XEN) ACPI: IRQ9 used by override. (XEN) Enabling APIC mode: Phys. Using 2 I/O APICs (XEN) ACPI: HPET id: 0x8086a301 base: 0xfed00000 (XEN) PCI: MCFG configuration 0: base e0000000 segment 0000 buses 00 - ff (XEN) PCI: MCFG area at e0000000 reserved in E820 (XEN) PCI: Using MCFG for segment 0000 bus 00-ff (XEN) [VT-D]Host address width 40 (XEN) [VT-D]found ACPI_DMAR_DRHD: (XEN) [VT-D] dmaru->address = fbffe000 (XEN) [VT-D]drhd->address = fbffe000 iommu->reg = ffff82c00021d000 (XEN) [VT-D]cap = c90780106f0462 ecap = f020fe (XEN) [VT-D] IOAPIC: 0000:f0:1f.7 (XEN) [VT-D] IOAPIC: 0000:00:13.0 (XEN) [VT-D] flags: INCLUDE_ALL (XEN) [VT-D]found ACPI_DMAR_RMRR: (XEN) [VT-D] endpoint: 0000:00:1d.0 (XEN) [VT-D] endpoint: 0000:00:1d.1 (XEN) [VT-D] endpoint: 0000:00:1d.2 (XEN) [VT-D] endpoint: 0000:00:1d.7 (XEN) [VT-D] endpoint: 0000:00:1a.0 (XEN) [VT-D] endpoint: 0000:00:1a.1 (XEN) [VT-D] endpoint: 0000:00:1a.2 (XEN) [VT-D] endpoint: 0000:00:1a.7 (XEN) [VT-D]found ACPI_DMAR_RMRR: (XEN) [VT-D] endpoint: 0000:00:1d.0 (XEN) [VT-D] endpoint: 0000:00:1d.1 (XEN) [VT-D] endpoint: 0000:00:1d.2 (XEN) [VT-D] endpoint: 0000:00:1d.7 (XEN) [VT-D] endpoint: 0000:00:1a.0 (XEN) [VT-D] endpoint: 0000:00:1a.1 (XEN) [VT-D] endpoint: 0000:00:1a.2 (XEN) [VT-D] endpoint: 0000:00:1a.7 (XEN) [VT-D]found ACPI_DMAR_ATSR: (XEN) [VT-D] atsru->all_ports: 0 (XEN) [VT-D] bridge: 0000:00:01.0 start=0 sec=1 sub=1 (XEN) [VT-D] bridge: 0000:00:03.0 start=0 sec=2 sub=2 (XEN) [VT-D] bridge: 0000:00:05.0 start=0 sec=3 sub=3 (XEN) [VT-D] bridge: 0000:00:07.0 start=0 sec=4 sub=4 (XEN) [VT-D] bridge: 0000:00:09.0 start=0 sec=5 sub=5 (XEN) Failed to get Error Log Address Range. (XEN) HEST: Table parsing has been initialized (XEN) Using ACPI (MADT) for SMP configuration information (XEN) SMP: Allowing 24 CPUs (8 hotplug CPUs) (XEN) IRQ limits: 48 GSI, 3040 MSI/MSI-X (XEN) CPU0: Intel machine check reporting enabled (XEN) Speculative mitigation facilities: (XEN) Hardware features: (XEN) Compiled-in support: SHADOW_PAGING (XEN) Xen settings: BTI-Thunk N/A, SPEC_CTRL: No, Other: (XEN) L1TF: believed vulnerable, maxphysaddr L1D 44, CPUID 40, Safe address 10000000000 (XEN) Support for HVM VMs: RSB EAGER_FPU (XEN) Support for PV VMs: RSB EAGER_FPU (XEN) XPTI (64-bit PV only): Dom0 enabled, DomU enabled (without PCID) (XEN) PV L1TF shadowing: Dom0 disabled, DomU enabled (XEN) Using scheduler: SMP Credit Scheduler rev2 (credit2) (XEN) Initializing Credit2 scheduler (XEN) load_precision_shift: 18 (XEN) load_window_shift: 30 (XEN) underload_balance_tolerance: 0 (XEN) overload_balance_tolerance: -3 (XEN) runqueues arrangement: socket (XEN) cap enforcement granularity: 10ms (XEN) load tracking window length 1073741824 ns (XEN) Adding cpu 0 to runqueue 0 (XEN) First cpu on runqueue, activating (XEN) Platform timer is 14.318MHz HPET (XEN) Detected 2266.768 MHz processor. (XEN) Initing memory sharing. (XEN) alt table ffff82d0804309b8 -> ffff82d0804318d0 (XEN) Intel VT-d iommu 0 supported page sizes: 4kB. (XEN) Intel VT-d Snoop Control enabled. (XEN) Intel VT-d Dom0 DMA Passthrough not enabled. (XEN) Intel VT-d Queued Invalidation enabled. (XEN) Intel VT-d Interrupt Remapping not enabled. (XEN) Intel VT-d Posted Interrupt not enabled. (XEN) Intel VT-d Shared EPT tables not enabled. (XEN) I/O virtualisation enabled (XEN) - Dom0 mode: Relaxed (XEN) Interrupt remapping disabled (XEN) nr_sockets: 3 (XEN) ENABLING IO-APIC IRQs (XEN) -> Using new ACK method (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1 (XEN) Allocated console ring of 128 KiB. (XEN) mwait-idle: MWAIT substates: 0x1120 (XEN) mwait-idle: v0.4.1 model 0x1a (XEN) mwait-idle: lapic_timer_reliable_states 0x2 (XEN) HPET: 0 timers usable for broadcast (4 total) (XEN) VMX: Supported advanced features: (XEN) - APIC MMIO access virtualisation (XEN) - APIC TPR shadow (XEN) - Extended Page Tables (EPT) (XEN) - Virtual-Processor Identifiers (VPID) (XEN) - Virtual NMI (XEN) - MSR direct-access bitmap (XEN) HVM: ASIDs enabled. (XEN) HVM: VMX enabled (XEN) HVM: Hardware Assisted Paging (HAP) detected (XEN) HVM: HAP page sizes: 4kB, 2MB (XEN) Adding cpu 1 to runqueue 0 (XEN) Adding cpu 2 to runqueue 0 (XEN) Adding cpu 3 to runqueue 0 (XEN) Adding cpu 4 to runqueue 0 (XEN) Adding cpu 5 to runqueue 0 (XEN) Adding cpu 6 to runqueue 0 (XEN) Adding cpu 7 to runqueue 0 (XEN) Adding cpu 8 to runqueue 1 (XEN) First cpu on runqueue, activating (XEN) Adding cpu 9 to runqueue 1 (XEN) Adding cpu 10 to runqueue 1 (XEN) Adding cpu 11 to runqueue 1 (XEN) Adding cpu 12 to runqueue 1 (XEN) Adding cpu 13 to runqueue 1 (XEN) Adding cpu 14 to runqueue 1 (XEN) Adding cpu 15 to runqueue 1 (XEN) Brought up 16 CPUs (XEN) build-id: aba2ec2f0b48776dac7a6e615962fec7240f6447 (XEN) ACPI sleep modes: S3 (XEN) VPMU: disabled (XEN) mcheck_poll: Machine check polling timer started. (XEN) Dom0 has maximum 816 PIRQs (XEN) NX (Execute Disable) protection active (XEN) *** Building a PVH Dom0 *** (XEN) [VT-D]d0:Hostbridge: skip 0000:00:00.0 map (XEN) Masked UR signaling on 0000:00:00.0 (XEN) Masked UR signaling on 0000:00:01.0 (XEN) Masked UR signaling on 0000:00:03.0 (XEN) Masked UR signaling on 0000:00:05.0 (XEN) Masked UR signaling on 0000:00:07.0 (XEN) Masked UR signaling on 0000:00:09.0 (XEN) [VT-D]d0:PCI: map 0000:00:13.0 (XEN) [VT-D]d0:PCIe: map 0000:00:14.0 (XEN) Masked VT-d error signaling on 0000:00:14.0 (XEN) [VT-D]d0:PCIe: map 0000:00:14.1 (XEN) [VT-D]d0:PCIe: map 0000:00:14.2 (XEN) [VT-D]d0:PCI: map 0000:00:14.3 (XEN) [VT-D]d0:PCIe: map 0000:00:16.0 (XEN) [VT-D]d0:PCIe: map 0000:00:16.1 (XEN) [VT-D]d0:PCIe: map 0000:00:16.2 (XEN) [VT-D]d0:PCIe: map 0000:00:16.3 (XEN) [VT-D]d0:PCIe: map 0000:00:16.4 (XEN) [VT-D]d0:PCIe: map 0000:00:16.5 (XEN) [VT-D]d0:PCIe: map 0000:00:16.6 (XEN) [VT-D]d0:PCIe: map 0000:00:16.7 (XEN) [VT-D]d0:PCI: map 0000:00:1a.0 (XEN) [VT-D]d0:PCI: map 0000:00:1a.1 (XEN) [VT-D]d0:PCI: map 0000:00:1a.2 (XEN) [VT-D]d0:PCI: map 0000:00:1a.7 (XEN) [VT-D]d0:PCI: map 0000:00:1d.0 (XEN) [VT-D]d0:PCI: map 0000:00:1d.1 (XEN) [VT-D]d0:PCI: map 0000:00:1d.2 (XEN) [VT-D]d0:PCI: map 0000:00:1d.7 (XEN) [VT-D]d0:PCI: map 0000:00:1f.0 (XEN) [VT-D]d0:PCI: map 0000:00:1f.2 (XEN) [VT-D]d0:PCI: map 0000:00:1f.3 (XEN) [VT-D]d0:PCIe: map 0000:01:00.0 (XEN) [VT-D]d0:PCIe: map 0000:01:00.1 (XEN) [VT-D]d0:PCIe: map 0000:03:00.0 (XEN) [VT-D]d0:PCIe: map 0000:03:00.1 (XEN) [VT-D]d0:PCI: map 0000:06:01.0 (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:00.0 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:00.1 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:02.0 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:02.1 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:02.4 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:02.5 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:03.0 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:03.1 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:03.2 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:03.4 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:04.0 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:04.1 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:04.2 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:04.3 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:05.0 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:05.1 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:05.2 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:05.3 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:06.0 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:06.1 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:06.2 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:06.3 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:00.0 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:00.1 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:02.0 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:02.1 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:02.4 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:02.5 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:03.0 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:03.1 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:03.2 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:03.4 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:04.0 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:04.1 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:04.2 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:04.3 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:05.0 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:05.1 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:05.2 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:05.3 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:06.0 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:06.1 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:06.2 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:06.3 map (XEN) [VT-D]iommu_enable_translation: iommu->reg = ffff82c00021d000 (XEN) Dom0 memory allocation stats: (XEN) order 3 allocations: 2 (XEN) order 4 allocations: 1 (XEN) order 5 allocations: 1 (XEN) order 6 allocations: 1 (XEN) order 7 allocations: 3 (XEN) order 8 allocations: 2 (XEN) order 9 allocations: 2 (XEN) order 10 allocations: 2 (XEN) order 11 allocations: 2 (XEN) order 12 allocations: 2 (XEN) order 13 allocations: 2 (XEN) order 14 allocations: 2 (XEN) order 15 allocations: 2 (XEN) order 16 allocations: 2 (XEN) order 17 allocations: 2 (XEN) order 18 allocations: 2 (XEN) WARNING: PVH is an experimental mode with limited functionality (XEN) Initial low memory virq threshold set at 0x4000 pages. (XEN) Scrubbing Free RAM in background (XEN) Std. Loglevel: All (XEN) Guest Loglevel: All (XEN) *************************************************** (XEN) WARNING: CONSOLE OUTPUT IS SYNCHRONOUS (XEN) This option is intended to aid ging of Xen by ensuring (XEN) that all output is synchronously delivered on the serial line. (XEN) However it can introduce SIGNIFICANT latencies and affect (XEN) timekeeping. It is NOT recommended for production use! (XEN) *************************************************** (XEN) Booted on L1TF-vulnerable hardware with SMT/Hyperthreading (XEN) enabled. Please assess your configuration and choose an (XEN) explicit 'smt=' setting. See XSA-273. (XEN) *************************************************** (XEN) Booted on MLPDS/MFBDS-vulnerable hardware with SMT/Hyperthreading (XEN) enabled. Mitigations will not be fully effective. Please (XEN) choose an explicit smt= setting. See XSA-297. (XEN) *************************************************** (XEN) 3... 2... 1... (XEN) Xen is relinquishing VGA console. (XEN) *** Serial input to DOM0 (type 'CTRL-a' three times to switch input) (XEN) Freed 500kB init memory ---<>--- Copyright (c) 1992-2018 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 12.0-RELEASE-p10 GENERIC amd64 FreeBSD clang version 6.0.1 (tags/RELEASE_601/final 335540) (based on LLVM 6.0.1) VT(vga): text 80x25 XEN: Hypervisor version 4.12 detected. CPU: Intel(R) Xeon(R) CPU E5520 @ 2.27GHz (2266.77-MHz K8-class CPU) Origin="GenuineIntel" Id=0x106a5 Family=0x6 Model=0x1a Stepping=5 Features=0x1fc3fbff Features2=0x80982201 AMD Features=0x28100800 AMD Features2=0x1 TSC: P-state invariant Hypervisor: Origin = "XenVMMXenVMM" real memory = 5378048000 (5128 MB) avail memory = 4099301376 (3909 MB) Event timer "LAPIC" quality 100 ACPI APIC Table: <080312 APIC1521> FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs FreeBSD/SMP: 1 package(s) x 4 core(s) random: unblocking device. Firmware Warning (ACPI): 32/64X length mismatch in FADT/Gpe0Block: 128/64 (20181003/tbfadt-748) ioapic1: Changing APIC ID to 9 ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-24 on motherboard Launching APs: 2 1 3 random: entropy device external interface kbd1 at kbdmux0 netmap: loaded module [ath_hal] loaded module_register_init: MOD_LOAD (vesa, 0xffffffff810f9a00, 0) error 19 nexus0 vtvga0: on motherboard cryptosoft0: on motherboard acpi0: on motherboard (XEN) d0: bind: m_gsi=9 g_gsi=9 acpi0: Power Button (fixed) cpu0: on acpi0 attimer0: port 0x40-0x43 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 atrtc0: port 0x70-0x71 irq 8 on acpi0 atrtc0: registered as a time-of-day clock, resolution 1.000000s (XEN) d0: bind: m_gsi=8 g_gsi=8 Event timer "RTC" frequency 32768 Hz quality 0 hpet0: iomem 0xfed03000-0xfed033ff on acpi0 hpet0: HPET never increments, disabling device_attach: hpet0 attach returned 6 Timecounter "ACPI-safe" frequency 3579545 Hz quality 850 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: at device 1.0 on pci0 pci1: on pcib1 igb0: port 0xdc00-0xdc1f mem 0xfbce0000-0xfbcfffff,0xfbcc0000-0xfbcdffff,0xfbc9c000-0xfbc9ffff irq 28 at device 0.0 on pci1 igb0: attach_pre capping queues at 8 igb0: using 1024 tx descriptors and 1024 rx descriptors igb0: msix_init qsets capped at 8 igb0: pxm cpus: 4 queue msgs: 9 admincnt: 1 igb0: using 4 rx queues 4 tx queues igb0: Using MSIX interrupts with 5 vectors igb0: allocated for 4 tx_queues igb0: allocated for 4 rx_queues igb0: Ethernet address: 00:25:90:36:fd:2e igb0: netmap queues/slots: TX 4/1024, RX 4/1024 igb1: port 0xd880-0xd89f mem 0xfbc60000-0xfbc7ffff,0xfbc40000-0xfbc5ffff,0xfbc1c000-0xfbc1ffff irq 40 at device 0.1 on pci1 igb1: attach_pre capping queues at 8 igb1: using 1024 tx descriptors and 1024 rx descriptors igb1: msix_init qsets capped at 8 igb1: pxm cpus: 4 queue msgs: 9 admincnt: 1 igb1: using 4 rx queues 4 tx queues igb1: Using MSIX interrupts with 5 vectors igb1: allocated for 4 tx_queues igb1: allocated for 4 rx_queues igb1: Ethernet address: 00:25:90:36:fd:2f igb1: netmap queues/slots: TX 4/1024, RX 4/1024 pcib2: at device 3.0 on pci0 pci2: on pcib2 pcib3: at device 5.0 on pci0 pci3: on pcib3 igb2: port 0xec00-0xec1f mem 0xfbde0000-0xfbdfffff,0xfbdc0000-0xfbddffff,0xfbd9c000-0xfbd9ffff irq 26 at device 0.0 on pci3 igb2: attach_pre capping queues at 8 igb2: using 1024 tx descriptors and 1024 rx descriptors igb2: msix_init qsets capped at 8 igb2: pxm cpus: 4 queue msgs: 9 admincnt: 1 igb2: using 4 rx queues 4 tx queues igb2: Using MSIX interrupts with 5 vectors igb2: allocated for 4 tx_queues igb2: allocated for 4 rx_queues igb2: Ethernet address: 00:25:90:09:c8:da igb2: netmap queues/slots: TX 4/1024, RX 4/1024 igb3: port 0xe880-0xe89f mem 0xfbd60000-0xfbd7ffff,0xfbd40000-0xfbd5ffff,0xfbd1c000-0xfbd1ffff irq 25 at device 0.1 on pci3 igb3: attach_pre capping queues at 8 igb3: using 1024 tx descriptors and 1024 rx descriptors igb3: msix_init qsets capped at 8 igb3: pxm cpus: 4 queue msgs: 9 admincnt: 1 igb3: using 4 rx queues 4 tx queues igb3: Using MSIX interrupts with 5 vectors igb3: allocated for 4 tx_queues igb3: allocated for 4 rx_queues igb3: Ethernet address: 00:25:90:09:c8:db igb3: netmap queues/slots: TX 4/1024, RX 4/1024 pcib4: at device 7.0 on pci0 pci4: on pcib4 pcib5: at device 9.0 on pci0 pci5: on pcib5 pci0: at device 20.0 (no driver attached) pci0: at device 20.1 (no driver attached) pci0: at device 20.2 (no driver attached) pci0: at device 20.3 (no driver attached) uhci0: port 0xcc00-0xcc1f irq 16 at device 26.0 on pci0 (XEN) d0: bind: m_gsi=16 g_gsi=16 uhci0: LegSup = 0x2f00 usbus0 on uhci0 usbus0: 12Mbps Full Speed USB v1.0 uhci1: port 0xc880-0xc89f irq 21 at device 26.1 on pci0 (XEN) d0: bind: m_gsi=21 g_gsi=21 uhci1: LegSup = 0x2f00 usbus1 on uhci1 usbus1: 12Mbps Full Speed USB v1.0 uhci2: port 0xc800-0xc81f irq 19 at device 26.2 on pci0 (XEN) d0: bind: m_gsi=19 g_gsi=19 uhci2: LegSup = 0x2f00 usbus2 on uhci2 usbus2: 12Mbps Full Speed USB v1.0 ehci0: mem 0xfbeda000-0xfbeda3ff irq 18 at device 26.7 on pci0 (XEN) d0: bind: m_gsi=18 g_gsi=18 usbus3: EHCI version 1.0 usbus3 on ehci0 usbus3: 480Mbps High Speed USB v2.0 uhci3: port 0xc480-0xc49f irq 23 at device 29.0 on pci0 (XEN) d0: bind: m_gsi=23 g_gsi=23 uhci3: LegSup = 0x2f00 usbus4 on uhci3 usbus4: 12Mbps Full Speed USB v1.0 uhci4: port 0xc400-0xc41f irq 19 at device 29.1 on pci0 uhci4: LegSup = 0x2f00 usbus5 on uhci4 usbus5: 12Mbps Full Speed USB v1.0 uhci5: port 0xc080-0xc09f irq 18 at device 29.2 on pci0 uhci5: LegSup = 0x2f00 usbus6 on uhci5 usbus6: 12Mbps Full Speed USB v1.0 ehci1: mem 0xfbed8000-0xfbed83ff irq 23 at device 29.7 on pci0 usbus7: EHCI version 1.0 usbus7 on ehci1 usbus7: 480Mbps High Speed USB v2.0 pcib6: at device 30.0 on pci0 pci6: on pcib6 vgapci0: mem 0xf9000000-0xf9ffffff,0xfaffc000-0xfaffffff,0xfb000000-0xfb7fffff irq 18 at device 1.0 on pci6 vgapci0: Boot video device isab0: at device 31.0 on pci0 isa0: on isab0 ahci0: port 0xb480-0xb487,0xc000-0xc003,0xbc00-0xbc07,0xb880-0xb883,0xb800-0xb81f mem 0xfbed6000-0xfbed67ff irq 19 at device 31.2 on pci0 ahci0: AHCI v1.20 with 6 3Gbps ports, Port Multiplier not supported ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 ahcich2: at channel 2 on ahci0 ahcich3: at channel 3 on ahci0 ahcich4: at channel 4 on ahci0 ahcich5: at channel 5 on ahci0 ahciem0: on ahci0 acpi_button0: on acpi0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 (XEN) d0: bind: m_gsi=4 g_gsi=4 xenpv0: on motherboard granttable0: on xenpv0 xen_et0: on xenpv0 Event timer "XENTIMER" frequency 1000000000 Hz quality 950 Timecounter "XENTIMER" frequency 1000000000 Hz quality 950 xen_et0: registered as a time-of-day clock, resolution 0.000001s xenstore0: on xenpv0 xsd_dev0: on xenpv0 evtchn0: on xenpv0 privcmd0: on xenpv0 gntdev0: on xenpv0 0: on xenpv0 xc0: on xenpv0 ipmi0: on isa0 ipmi0: KCS mode found at io 0xca2 alignment 0x1 on isa orm0: at iomem 0xc0000-0xc7fff,0xc8000-0xc8fff,0xc9000-0xc9fff,0xca000-0xcafff pnpid ORM0000 on isa0 vga0: at port 0x3d0-0x3db iomem 0xb8000-0xbffff pnpid PNP0900 on isa0 atkbdc0: at port 0x60,0x64 on isa0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 (XEN) d0: bind: m_gsi=1 g_gsi=1 atkbd0: [GIANT-LOCKED] atkbdc0: non-PNP ISA device will be removed from GENERIC in FreeBSD 12. Timecounters tick every 10.000 msec ipmi0: IPMI device rev. 1, firmware rev. 3.13, version 2.0, device support mask 0xbf ipmi0: Number of channels 2 ipmi0: Attached watchdog ipmi0: Establishing power cycle handler ugen0.1: at usbus0 ugen2.1: at usbus2 uhub0: on usbus0 ugen3.1: at usbus3 uhub1: ugen5.1: at usbus5 ugen6.1: at usbus6 on usbus2 uhub3: on usbus6 ugen1.1: at usbus1 uhub4: on usbus5 uhub5: on usbus1 ugen4.1: at usbus4 ugen7.1: at usbus7 uhub2: on usbus3 uhub6: on usbus4 uhub7: on usbus7 uhub0: 2 ports with 2 removable, self powered uhub3: 2 ports with 2 removable, self powered uhub4: 2 ports with 2 removable, self powered uhub1: 2 ports with 2 removable, self powered uhub5: 2 ports with 2 removable, self powered uhub6: 2 ports with 2 removable, self powered uhub2: 6 ports with 6 removable, self powered uhub7: 6 ports with 6 removable, self powered usb_alloc_device: set address 2 failed (USB_ERR_TIMEOUT, ignored) usbd_setup_device_desc: getting device descriptor at addr 2 failed, USB_ERR_TIMEOUT usbd_req_re_enumerate: addr=2, set address failed! (USB_ERR_TIMEOUT, ignored) usbd_setup_device_desc: getting device descriptor at addr 2 failed, USB_ERR_TIMEOUT usbd_req_re_enumerate: addr=2, set address failed! (USB_ERR_TIMEOUT, ignored) usbd_setup_device_desc: getting device descriptor at addr 2 failed, USB_ERR_TIMEOUT usbd_req_re_enumerate: addr=2, set address failed! (USB_ERR_TIMEOUT, ignored) usbd_setup_device_desc: getting device descriptor at addr 2 failed, USB_ERR_TIMEOUT usbd_req_re_enumerate: addr=2, set address failed! (USB_ERR_TIMEOUT, ignored) usbd_setup_device_desc: getting device descriptor at addr 2 failed, USB_ERR_TIMEOUT ugen1.2: at usbus1 (disconnected) uhub_reattach_port: could not allocate new device ahcich3: Timeout on slot 3 port 0 ahcich3: is 00000002 cs 00000000 ss 00000000 rs 00000008 tfd 50 serr 00000000 cmd 0004c317 (aprobe3:ahcich3:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00 ahcich0: Timeout on slot 3 port 0 (aprobe3:ahcich3:0:0:0): CAM status: Command timeout ahcich0: (aprobe3:ahcich3:0:0:0): Retrying command, 0 more tries remain is 00000002 cs 00000000 ss 00000000 rs 00000008 tfd 50 serr 00000000 cmd 0004c317 (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00 ahcich1: Timeout on slot 3 port 0 (aprobe0:ahcich0:0:0:0): CAM status: Command timeout ahcich1: is 00000002 cs 00000000 ss 00000000 rs 00000008 tfd 50 serr 00000000 cmd 0004c317 (aprobe0:ahcich0:0:0:0): Retrying command, 0 more tries remain ahcich2: Timeout on slot 3 port 0 ahcich2: is 00000002 cs 00000000 ss 00000000 rs 00000008 tfd 50 serr 00000000 cmd 0004c317 (aprobe1:ahcich1:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00 (aprobe1:ahcich1:0:0:0): CAM status: Command timeout (aprobe1:ahcich1:0:0:0): Retrying command, 0 more tries remain (aprobe2:ahcich2:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00 (aprobe2:ahcich2:0:0:0): CAM status: Command timeout (aprobe2:ahcich2:0:0:0): Retrying command, 0 more tries remain ahcich4: Timeout on slot 3 port 0 ahcich4: is 00000002 cs 00000000 ss 00000000 rs 00000008 tfd 50 serr 00000000 cmd 0004c317 (aprobe4:ahcich4:0:0:0): ATAPI_IDENTIFY. ACB: a1 00 00 00 00 40 00 00 00 00 00 00 (aprobe4:ahcich4:0:0:0): CAM status: Command timeout (aprobe4:ahcich4:0:0:0): Retrying command, 0 more tries remain run_interrupt_driven_hooks: still waiting after 60 seconds for xpt_register_async From owner-freebsd-xen@freebsd.org Sat Oct 12 15:31:47 2019 Return-Path: Delivered-To: freebsd-xen@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9E1AE1392DA for ; Sat, 12 Oct 2019 15:31:47 +0000 (UTC) (envelope-from buhrow@lothlorien.nfbcal.org) Received: from lothlorien.nfbcal.org (ns.NFBCAL.ORG [157.22.230.125]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "nfbcal.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46r81t14Z8z4D6N for ; Sat, 12 Oct 2019 15:31:45 +0000 (UTC) (envelope-from buhrow@lothlorien.nfbcal.org) Received: from lothlorien.nfbcal.org (localhost [127.0.0.1]) by lothlorien.nfbcal.org (8.15.2/8.14.1-NFBNETBSD) with ESMTPS id x9CFVbcS025650 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sat, 12 Oct 2019 08:31:37 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.97.3 at lothlorien.nfbcal.org Received: (from buhrow@localhost) by lothlorien.nfbcal.org (8.15.2/8.12.11) id x9CFVaGT014403; Sat, 12 Oct 2019 08:31:36 -0700 (PDT) Message-Id: <201910121531.x9CFVaGT014403@lothlorien.nfbcal.org> From: Brian Buhrow Date: Sat, 12 Oct 2019 08:31:36 -0700 In-Reply-To: <20191012095900.GA14005@Air-de-Roger> X-Mailer: Mail User's Shell (7.2.6 beta(4.pl1)+dynamic 20000103) To: Roger Pau =?iso-8859-1?Q?Monn=E9?= Subject: Re: i/o is very slow on FreeBSD dom0 with Xen-4.12 and Freebsd-12 Cc: , buhrow@nfbcal.org X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (lothlorien.nfbcal.org [127.0.0.1]); Sat, 12 Oct 2019 08:31:37 -0700 (PDT) X-Rspamd-Queue-Id: 46r81t14Z8z4D6N X-Spamd-Bar: + Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of buhrow@lothlorien.nfbcal.org designates 157.22.230.125 as permitted sender) smtp.mailfrom=buhrow@lothlorien.nfbcal.org X-Spamd-Result: default: False [1.56 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.96)[-0.956,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+a:ns.nfbcal.org:c]; NEURAL_HAM_LONG(-0.85)[-0.853,0]; R_BAD_CTE_7BIT(3.50)[7bit]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[nfbcal.org]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; IP_SCORE(-0.13)[ipnet: 157.22.0.0/16(-0.33), asn: 7091(-0.26), country: US(-0.05)]; FORGED_SENDER(0.30)[buhrow@nfbcal.org,buhrow@lothlorien.nfbcal.org]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:7091, ipnet:157.22.0.0/16, country:US]; FROM_NEQ_ENVFROM(0.00)[buhrow@nfbcal.org,buhrow@lothlorien.nfbcal.org]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Oct 2019 15:31:47 -0000 Hello. Here are the logs with remapping enabled, plus the i, m and z xen console commands. Here's hoping you can provide a fix soon! Or, perhaps, a clue on how to fix. -thanks -Brian boot_serial="YES" # -h: Use serial console comconsole_speed="115200" # Set the current serial console speed comconsole_port="0x2f8" # Set the current serial console port #console="vidconsole" # A comma separated list of console(s) console="comconsole" # A comma separated list of console(s) geom_mirror_load="YES" # RAID1 disk driver (see gmirror(8)) ahci_load="YES" # ahci driver ipmi_load="YES" # Ripmi driver if_tap_load="YES" # Load bridge driver. if_vlan_load="YES" # Load vlan driver. # Turn on Xen (BB 10/08/2019) xen_kernel="/boot/xen" xen_cmdline="dom0_mem=4096m dom0_max_vcpus=4 dom0=pvh,verbose console=com2,vga iommu=verbose,debug sync_console=true com2=115200,8n1 guest_loglvl=all loglvl=all" (XEN) Xen version 4.12.1 (buhrow@) (FreeBSD clang version 6.0.1 (tags/RELEASE_601/final 335540) (based on LLVM 6.0.1)) debug=n Thu Oct 3 11:34:20 PDT 2019 (XEN) Latest ChangeSet: (XEN) Console output is synchronous. (XEN) Bootloader: FreeBSD Loader (XEN) Command line: dom0_mem=4096m dom0_max_vcpus=4 dom0=pvh,verbose console=com2,vga iommu=verbose,debug sync_console=true com2=115200,8n1 guest_loglvl=all loglvl=all (XEN) Xen image load base address: 0 (XEN) Video information: (XEN) VGA is text mode 80x25, font 8x16 (XEN) VBE/DDC methods: none; EDID transfer time: 0 seconds (XEN) EDID info not retrieved because no DDC retrieval method detected (XEN) Disc information: (XEN) Found 4 MBR signatures (XEN) Found 4 EDD information structures (XEN) Xen-e820 RAM map: (XEN) 0000000000000000 - 0000000000098400 (usable) (XEN) 0000000000098400 - 00000000000a0000 (reserved) (XEN) 00000000000e0000 - 0000000000100000 (reserved) (XEN) 0000000000100000 - 00000000bf780000 (usable) (XEN) 00000000bf78e000 - 00000000bf790000 type 9 (XEN) 00000000bf790000 - 00000000bf79e000 (ACPI data) (XEN) 00000000bf79e000 - 00000000bf7d0000 (ACPI NVS) (XEN) 00000000bf7d0000 - 00000000bf7e0000 (reserved) (XEN) 00000000bf7ec000 - 00000000c0000000 (reserved) (XEN) 00000000e0000000 - 00000000f0000000 (reserved) (XEN) 00000000fee00000 - 00000000fee01000 (reserved) (XEN) 00000000ffc00000 - 0000000100000000 (reserved) (XEN) 0000000100000000 - 0000001840000000 (usable) (XEN) New Xen image base address: 0xbf000000 (XEN) ACPI: RSDP 000FACE0, 0024 (r2 ACPIAM) (XEN) ACPI: XSDT BF790100, 008C (r1 SMCI 20120803 MSFT 97) (XEN) ACPI: FACP BF790290, 00F4 (r3 080312 FACP1521 20120803 MSFT 97) (XEN) ACPI: DSDT BF7906A0, 6580 (r1 10600 10600000 0 INTL 20051117) (XEN) ACPI: FACS BF79E000, 0040 (XEN) ACPI: APIC BF790390, 011E (r1 080312 APIC1521 20120803 MSFT 97) (XEN) ACPI: MCFG BF7904B0, 003C (r1 080312 OEMMCFG 20120803 MSFT 97) (XEN) ACPI: SLIT BF7904F0, 0030 (r1 080312 OEMSLIT 20120803 MSFT 97) (XEN) ACPI: OEMB BF79E040, 0086 (r1 080312 OEMB1521 20120803 MSFT 97) (XEN) ACPI: SRAT BF79A6A0, 01D0 (r1 080312 OEMSRAT 1 INTL 1) (XEN) ACPI: HPET BF79A870, 0038 (r1 080312 OEMHPET 20120803 MSFT 97) (XEN) ACPI: DMAR BF79E0D0, 0130 (r1 AMI OEMDMAR 1 MSFT 97) (XEN) ACPI: SSDT BF7A1710, 0363 (r1 DpgPmm CpuPm 12 INTL 20051117) (XEN) ACPI: EINJ BF79A8B0, 0130 (r1 AMIER AMI_EINJ 20120803 MSFT 97) (XEN) ACPI: BERT BF79AA40, 0030 (r1 AMIER AMI_BERT 20120803 MSFT 97) (XEN) ACPI: ERST BF79AA70, 01B0 (r1 AMIER AMI_ERST 20120803 MSFT 97) (XEN) ACPI: HEST BF79AC20, 00A8 (r1 AMIER ABC_HEST 20120803 MSFT 97) (XEN) System RAM: 98295MB (100654176kB) (XEN) SRAT: PXM 0 -> APIC 00 -> Node 0 (XEN) SRAT: PXM 0 -> APIC 02 -> Node 0 (XEN) SRAT: PXM 0 -> APIC 04 -> Node 0 (XEN) SRAT: PXM 0 -> APIC 06 -> Node 0 (XEN) SRAT: PXM 0 -> APIC 01 -> Node 0 (XEN) SRAT: PXM 0 -> APIC 03 -> Node 0 (XEN) SRAT: PXM 0 -> APIC 05 -> Node 0 (XEN) SRAT: PXM 0 -> APIC 07 -> Node 0 (XEN) SRAT: PXM 1 -> APIC 10 -> Node 1 (XEN) SRAT: PXM 1 -> APIC 12 -> Node 1 (XEN) SRAT: PXM 1 -> APIC 14 -> Node 1 (XEN) SRAT: PXM 1 -> APIC 16 -> Node 1 (XEN) SRAT: PXM 1 -> APIC 11 -> Node 1 (XEN) SRAT: PXM 1 -> APIC 13 -> Node 1 (XEN) SRAT: PXM 1 -> APIC 15 -> Node 1 (XEN) SRAT: PXM 1 -> APIC 17 -> Node 1 (XEN) SRAT: Node 0 PXM 0 0-a0000 (XEN) SRAT: Node 0 PXM 0 100000-c0000000 (XEN) SRAT: Node 0 PXM 0 100000000-c40000000 (XEN) SRAT: Node 1 PXM 1 c40000000-1840000000 (XEN) NUMA: Allocated memnodemap from 183e1b7000 - 183e1d0000 (XEN) NUMA: Using 8 for the hash shift. (XEN) Domain heap initialised DMA width 32 bits (XEN) CPU Vendor: Intel, Family 6 (0x6), Model 26 (0x1a), Stepping 5 (raw 000106a5) (XEN) found SMP MP-table at 000ff780 (XEN) DMI present. (XEN) Using APIC driver default (XEN) ACPI: PM-Timer IO Port: 0x808 (32 bits) (XEN) ACPI: SLEEP INFO: pm1x_cnt[1:804,1:0], pm1x_evt[1:800,1:0] (XEN) ACPI: wakeup_vec[bf79e00c], vec_size[20] (XEN) ACPI: Local APIC address 0xfee00000 (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled) (XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x10] enabled) (XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x02] enabled) (XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x12] enabled) (XEN) ACPI: LAPIC (acpi_id[0x05] lapic_id[0x04] enabled) (XEN) ACPI: LAPIC (acpi_id[0x06] lapic_id[0x14] enabled) (XEN) ACPI: LAPIC (acpi_id[0x07] lapic_id[0x06] enabled) (XEN) ACPI: LAPIC (acpi_id[0x08] lapic_id[0x16] enabled) (XEN) ACPI: LAPIC (acpi_id[0x09] lapic_id[0x01] enabled) (XEN) ACPI: LAPIC (acpi_id[0x0a] lapic_id[0x11] enabled) (XEN) ACPI: LAPIC (acpi_id[0x0b] lapic_id[0x03] enabled) (XEN) ACPI: LAPIC (acpi_id[0x0c] lapic_id[0x13] enabled) (XEN) ACPI: LAPIC (acpi_id[0x0d] lapic_id[0x05] enabled) (XEN) ACPI: LAPIC (acpi_id[0x0e] lapic_id[0x15] enabled) (XEN) ACPI: LAPIC (acpi_id[0x0f] lapic_id[0x07] enabled) (XEN) ACPI: LAPIC (acpi_id[0x10] lapic_id[0x17] enabled) (XEN) ACPI: LAPIC (acpi_id[0x11] lapic_id[0x90] disabled) (XEN) ACPI: LAPIC (acpi_id[0x12] lapic_id[0x91] disabled) (XEN) ACPI: LAPIC (acpi_id[0x13] lapic_id[0x92] disabled) (XEN) ACPI: LAPIC (acpi_id[0x14] lapic_id[0x93] disabled) (XEN) ACPI: LAPIC (acpi_id[0x15] lapic_id[0x94] disabled) (XEN) ACPI: LAPIC (acpi_id[0x16] lapic_id[0x95] disabled) (XEN) ACPI: LAPIC (acpi_id[0x17] lapic_id[0x96] disabled) (XEN) ACPI: LAPIC (acpi_id[0x18] lapic_id[0x97] disabled) (XEN) ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) (XEN) Overriding APIC driver with bigsmp (XEN) ACPI: IOAPIC (id[0x08] address[0xfec00000] gsi_base[0]) (XEN) IOAPIC[0]: apic_id 8, version 32, address 0xfec00000, GSI 0-23 (XEN) ACPI: IOAPIC (id[0x09] address[0xfec8a000] gsi_base[24]) (XEN) IOAPIC[1]: apic_id 9, version 32, address 0xfec8a000, GSI 24-47 (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) (XEN) ACPI: IRQ0 used by override. (XEN) ACPI: IRQ2 used by override. (XEN) ACPI: IRQ9 used by override. (XEN) Enabling APIC mode: Phys. Using 2 I/O APICs (XEN) ACPI: HPET id: 0x8086a301 base: 0xfed00000 (XEN) PCI: MCFG configuration 0: base e0000000 segment 0000 buses 00 - ff (XEN) PCI: MCFG area at e0000000 reserved in E820 (XEN) PCI: Using MCFG for segment 0000 bus 00-ff (XEN) [VT-D]Host address width 40 (XEN) [VT-D]found ACPI_DMAR_DRHD: (XEN) [VT-D] dmaru->address = fbffe000 (XEN) [VT-D]drhd->address = fbffe000 iommu->reg = ffff82c00021d000 (XEN) [VT-D]cap = c90780106f0462 ecap = f020fe (XEN) [VT-D] IOAPIC: 0000:f0:1f.7 (XEN) [VT-D] IOAPIC: 0000:00:13.0 (XEN) [VT-D] flags: INCLUDE_ALL (XEN) [VT-D]found ACPI_DMAR_RMRR: (XEN) [VT-D] endpoint: 0000:00:1d.0 (XEN) [VT-D] endpoint: 0000:00:1d.1 (XEN) [VT-D] endpoint: 0000:00:1d.2 (XEN) [VT-D] endpoint: 0000:00:1d.7 (XEN) [VT-D] endpoint: 0000:00:1a.0 (XEN) [VT-D] endpoint: 0000:00:1a.1 (XEN) [VT-D] endpoint: 0000:00:1a.2 (XEN) [VT-D] endpoint: 0000:00:1a.7 (XEN) [VT-D]found ACPI_DMAR_RMRR: (XEN) [VT-D] endpoint: 0000:00:1d.0 (XEN) [VT-D] endpoint: 0000:00:1d.1 (XEN) [VT-D] endpoint: 0000:00:1d.2 (XEN) [VT-D] endpoint: 0000:00:1d.7 (XEN) [VT-D] endpoint: 0000:00:1a.0 (XEN) [VT-D] endpoint: 0000:00:1a.1 (XEN) [VT-D] endpoint: 0000:00:1a.2 (XEN) [VT-D] endpoint: 0000:00:1a.7 (XEN) [VT-D]found ACPI_DMAR_ATSR: (XEN) [VT-D] atsru->all_ports: 0 (XEN) [VT-D] bridge: 0000:00:01.0 start=0 sec=1 sub=1 (XEN) [VT-D] bridge: 0000:00:03.0 start=0 sec=2 sub=2 (XEN) [VT-D] bridge: 0000:00:05.0 start=0 sec=3 sub=3 (XEN) [VT-D] bridge: 0000:00:07.0 start=0 sec=4 sub=4 (XEN) [VT-D] bridge: 0000:00:09.0 start=0 sec=5 sub=5 (XEN) Failed to get Error Log Address Range. (XEN) HEST: Table parsing has been initialized (XEN) Using ACPI (MADT) for SMP configuration information (XEN) SMP: Allowing 24 CPUs (8 hotplug CPUs) (XEN) IRQ limits: 48 GSI, 3040 MSI/MSI-X (XEN) CPU0: Intel machine check reporting enabled (XEN) Speculative mitigation facilities: (XEN) Hardware features: (XEN) Compiled-in support: SHADOW_PAGING (XEN) Xen settings: BTI-Thunk N/A, SPEC_CTRL: No, Other: (XEN) L1TF: believed vulnerable, maxphysaddr L1D 44, CPUID 40, Safe address 10000000000 (XEN) Support for HVM VMs: RSB EAGER_FPU (XEN) Support for PV VMs: RSB EAGER_FPU (XEN) XPTI (64-bit PV only): Dom0 enabled, DomU enabled (without PCID) (XEN) PV L1TF shadowing: Dom0 disabled, DomU enabled (XEN) Using scheduler: SMP Credit Scheduler rev2 (credit2) (XEN) Initializing Credit2 scheduler (XEN) load_precision_shift: 18 (XEN) load_window_shift: 30 (XEN) underload_balance_tolerance: 0 (XEN) overload_balance_tolerance: -3 (XEN) runqueues arrangement: socket (XEN) cap enforcement granularity: 10ms (XEN) load tracking window length 1073741824 ns (XEN) Adding cpu 0 to runqueue 0 (XEN) First cpu on runqueue, activating (XEN) Platform timer is 14.318MHz HPET (XEN) Detected 2266.768 MHz processor. (XEN) Initing memory sharing. (XEN) alt table ffff82d0804309b8 -> ffff82d0804318d0 (XEN) Intel VT-d iommu 0 supported page sizes: 4kB. (XEN) Intel VT-d Snoop Control enabled. (XEN) Intel VT-d Dom0 DMA Passthrough not enabled. (XEN) Intel VT-d Queued Invalidation enabled. (XEN) Intel VT-d Interrupt Remapping enabled. (XEN) Intel VT-d Posted Interrupt not enabled. (XEN) Intel VT-d Shared EPT tables not enabled. (XEN) I/O virtualisation enabled (XEN) - Dom0 mode: Relaxed (XEN) Interrupt remapping enabled (XEN) nr_sockets: 3 (XEN) ENABLING IO-APIC IRQs (XEN) -> Using new ACK method (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1 (XEN) Allocated console ring of 128 KiB. (XEN) mwait-idle: MWAIT substates: 0x1120 (XEN) mwait-idle: v0.4.1 model 0x1a (XEN) mwait-idle: lapic_timer_reliable_states 0x2 (XEN) HPET: 0 timers usable for broadcast (4 total) (XEN) VMX: Supported advanced features: (XEN) - APIC MMIO access virtualisation (XEN) - APIC TPR shadow (XEN) - Extended Page Tables (EPT) (XEN) - Virtual-Processor Identifiers (VPID) (XEN) - Virtual NMI (XEN) - MSR direct-access bitmap (XEN) HVM: ASIDs enabled. (XEN) HVM: VMX enabled (XEN) HVM: Hardware Assisted Paging (HAP) detected (XEN) HVM: HAP page sizes: 4kB, 2MB (XEN) Adding cpu 1 to runqueue 0 (XEN) Adding cpu 2 to runqueue 0 (XEN) Adding cpu 3 to runqueue 0 (XEN) Adding cpu 4 to runqueue 0 (XEN) Adding cpu 5 to runqueue 0 (XEN) Adding cpu 6 to runqueue 0 (XEN) Adding cpu 7 to runqueue 0 (XEN) Adding cpu 8 to runqueue 1 (XEN) First cpu on runqueue, activating (XEN) Adding cpu 9 to runqueue 1 (XEN) Adding cpu 10 to runqueue 1 (XEN) Adding cpu 11 to runqueue 1 (XEN) Adding cpu 12 to runqueue 1 (XEN) Adding cpu 13 to runqueue 1 (XEN) Adding cpu 14 to runqueue 1 (XEN) Adding cpu 15 to runqueue 1 (XEN) Brought up 16 CPUs (XEN) build-id: aba2ec2f0b48776dac7a6e615962fec7240f6447 (XEN) ACPI sleep modes: S3 (XEN) VPMU: disabled (XEN) mcheck_poll: Machine check polling timer started. (XEN) Dom0 has maximum 816 PIRQs (XEN) NX (Execute Disable) protection active (XEN) *** Building a PVH Dom0 *** (XEN) [VT-D]d0:Hostbridge: skip 0000:00:00.0 map (XEN) Masked UR signaling on 0000:00:00.0 (XEN) Masked UR signaling on 0000:00:01.0 (XEN) Masked UR signaling on 0000:00:03.0 (XEN) Masked UR signaling on 0000:00:05.0 (XEN) Masked UR signaling on 0000:00:07.0 (XEN) Masked UR signaling on 0000:00:09.0 (XEN) [VT-D]d0:PCI: map 0000:00:13.0 (XEN) [VT-D]d0:PCIe: map 0000:00:14.0 (XEN) Masked VT-d error signaling on 0000:00:14.0 (XEN) [VT-D]d0:PCIe: map 0000:00:14.1 (XEN) [VT-D]d0:PCIe: map 0000:00:14.2 (XEN) [VT-D]d0:PCI: map 0000:00:14.3 (XEN) [VT-D]d0:PCIe: map 0000:00:16.0 (XEN) [VT-D]d0:PCIe: map 0000:00:16.1 (XEN) [VT-D]d0:PCIe: map 0000:00:16.2 (XEN) [VT-D]d0:PCIe: map 0000:00:16.3 (XEN) [VT-D]d0:PCIe: map 0000:00:16.4 (XEN) [VT-D]d0:PCIe: map 0000:00:16.5 (XEN) [VT-D]d0:PCIe: map 0000:00:16.6 (XEN) [VT-D]d0:PCIe: map 0000:00:16.7 (XEN) [VT-D]d0:PCI: map 0000:00:1a.0 (XEN) [VT-D]d0:PCI: map 0000:00:1a.1 (XEN) [VT-D]d0:PCI: map 0000:00:1a.2 (XEN) [VT-D]d0:PCI: map 0000:00:1a.7 (XEN) [VT-D]d0:PCI: map 0000:00:1d.0 (XEN) [VT-D]d0:PCI: map 0000:00:1d.1 (XEN) [VT-D]d0:PCI: map 0000:00:1d.2 (XEN) [VT-D]d0:PCI: map 0000:00:1d.7 (XEN) [VT-D]d0:PCI: map 0000:00:1f.0 (XEN) [VT-D]d0:PCI: map 0000:00:1f.2 (XEN) [VT-D]d0:PCI: map 0000:00:1f.3 (XEN) [VT-D]d0:PCIe: map 0000:01:00.0 (XEN) [VT-D]d0:PCIe: map 0000:01:00.1 (XEN) [VT-D]d0:PCIe: map 0000:03:00.0 (XEN) [VT-D]d0:PCIe: map 0000:03:00.1 (XEN) [VT-D]d0:PCI: map 0000:06:01.0 (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:00.0 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:00.1 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:02.0 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:02.1 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:02.4 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:02.5 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:03.0 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:03.1 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:03.2 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:03.4 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:04.0 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:04.1 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:04.2 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:04.3 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:05.0 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:05.1 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:05.2 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:05.3 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:06.0 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:06.1 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:06.2 map (XEN) [VT-D]d0:Hostbridge: skip 0000:fe:06.3 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:00.0 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:00.1 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:02.0 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:02.1 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:02.4 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:02.5 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:03.0 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:03.1 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:03.2 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:03.4 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:04.0 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:04.1 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:04.2 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:04.3 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:05.0 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:05.1 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:05.2 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:05.3 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:06.0 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:06.1 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:06.2 map (XEN) [VT-D]d0:Hostbridge: skip 0000:ff:06.3 map (XEN) [VT-D]iommu_enable_translation: iommu->reg = ffff82c00021d000 (XEN) Dom0 memory allocation stats: (XEN) order 3 allocations: 2 (XEN) order 4 allocations: 1 (XEN) order 5 allocations: 1 (XEN) order 6 allocations: 1 (XEN) order 7 allocations: 3 (XEN) order 8 allocations: 2 (XEN) order 9 allocations: 2 (XEN) order 10 allocations: 2 (XEN) order 11 allocations: 2 (XEN) order 12 allocations: 2 (XEN) order 13 allocations: 2 (XEN) order 14 allocations: 2 (XEN) order 15 allocations: 2 (XEN) order 16 allocations: 2 (XEN) order 17 allocations: 2 (XEN) order 18 allocations: 2 (XEN) WARNING: PVH is an experimental mode with limited functionality (XEN) Initial low memory virq threshold set at 0x4000 pages. (XEN) Scrubbing Free RAM in background (XEN) Std. Loglevel: All (XEN) Guest Loglevel: All (XEN) *************************************************** (XEN) WARNING: CONSOLE OUTPUT IS SYNCHRONOUS (XEN) This option is intended to aid debugging of Xen by ensuring (XEN) that all output is synchronously delivered on the serial line. (XEN) However it can introduce SIGNIFICANT latencies and affect (XEN) timekeeping. It is NOT recommended for production use! (XEN) *************************************************** (XEN) Booted on L1TF-vulnerable hardware with SMT/Hyperthreading (XEN) enabled. Please assess your configuration and choose an (XEN) explicit 'smt=' setting. See XSA-273. (XEN) *************************************************** (XEN) Booted on MLPDS/MFBDS-vulnerable hardware with SMT/Hyperthreading (XEN) enabled. Mitigations will not be fully effective. Please (XEN) choose an explicit smt= setting. See XSA-297. (XEN) *************************************************** (XEN) 3... 2... 1... (XEN) Xen is relinquishing VGA console. (XEN) *** Serial input to DOM0 (type 'CTRL-a' three times to switch input) (XEN) Freed 500kB init memory ---<>--- Copyright (c) 1992-2018 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 12.0-RELEASE-p10 GENERIC amd64 FreeBSD clang version 6.0.1 (tags/RELEASE_601/final 335540) (based on LLVM 6.0.1) VT(vga): text 80x25 XEN: Hypervisor version 4.12 detected. CPU: Intel(R) Xeon(R) CPU E5520 @ 2.27GHz (2266.77-MHz K8-class CPU) Origin="GenuineIntel" Id=0x106a5 Family=0x6 Model=0x1a Stepping=5 Features=0x1fc3fbff Features2=0x80982201 AMD Features=0x28100800 AMD Features2=0x1 TSC: P-state invariant Hypervisor: Origin = "XenVMMXenVMM" real memory = 5378048000 (5128 MB) avail memory = 4099301376 (3909 MB) Event timer "LAPIC" quality 100 ACPI APIC Table: <080312 APIC1521> FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs FreeBSD/SMP: 1 package(s) x 4 core(s) random: unblocking device. Firmware Warning (ACPI): 32/64X length mismatch in FADT/Gpe0Block: 128/64 (20181003/tbfadt-748) ioapic1: Changing APIC ID to 9 ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-24 on motherboard Launching APs: 2 1 3 random: entropy device external interface kbd1 at kbdmux0 netmap: loaded module [ath_hal] loaded module_register_init: MOD_LOAD (vesa, 0xffffffff810f9a00, 0) error 19 nexus0 vtvga0: on motherboard cryptosoft0: on motherboard acpi0: on motherboard (XEN) d0: bind: m_gsi=9 g_gsi=9 acpi0: Power Button (fixed) cpu0: on acpi0 attimer0: port 0x40-0x43 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 atrtc0: port 0x70-0x71 irq 8 on acpi0 atrtc0: registered as a time-of-day clock, resolution 1.000000s (XEN) d0: bind: m_gsi=8 g_gsi=8 Event timer "RTC" frequency 32768 Hz quality 0 hpet0: iomem 0xfed03000-0xfed033ff on acpi0 hpet0: HPET never increments, disabling device_attach: hpet0 attach returned 6 Timecounter "ACPI-safe" frequency 3579545 Hz quality 850 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: at device 1.0 on pci0 pci1: on pcib1 igb0: port 0xdc00-0xdc1f mem 0xfbce0000-0xfbcfffff,0xfbcc0000-0xfbcdffff,0xfbc9c000-0xfbc9ffff irq 28 at device 0.0 on pci1 igb0: attach_pre capping queues at 8 igb0: using 1024 tx descriptors and 1024 rx descriptors igb0: msix_init qsets capped at 8 igb0: pxm cpus: 4 queue msgs: 9 admincnt: 1 igb0: using 4 rx queues 4 tx queues igb0: Using MSIX interrupts with 5 vectors igb0: allocated for 4 tx_queues igb0: allocated for 4 rx_queues igb0: Ethernet address: 00:25:90:36:fd:2e igb0: netmap queues/slots: TX 4/1024, RX 4/1024 igb1: port 0xd880-0xd89f mem 0xfbc60000-0xfbc7ffff,0xfbc40000-0xfbc5ffff,0xfbc1c000-0xfbc1ffff irq 40 at device 0.1 on pci1 igb1: attach_pre capping queues at 8 igb1: using 1024 tx descriptors and 1024 rx descriptors igb1: msix_init qsets capped at 8 igb1: pxm cpus: 4 queue msgs: 9 admincnt: 1 igb1: using 4 rx queues 4 tx queues igb1: Using MSIX interrupts with 5 vectors igb1: allocated for 4 tx_queues igb1: allocated for 4 rx_queues igb1: Ethernet address: 00:25:90:36:fd:2f igb1: netmap queues/slots: TX 4/1024, RX 4/1024 pcib2: at device 3.0 on pci0 pci2: on pcib2 pcib3: at device 5.0 on pci0 pci3: on pcib3 igb2: port 0xec00-0xec1f mem 0xfbde0000-0xfbdfffff,0xfbdc0000-0xfbddffff,0xfbd9c000-0xfbd9ffff irq 26 at device 0.0 on pci3 igb2: attach_pre capping queues at 8 igb2: using 1024 tx descriptors and 1024 rx descriptors igb2: msix_init qsets capped at 8 igb2: pxm cpus: 4 queue msgs: 9 admincnt: 1 igb2: using 4 rx queues 4 tx queues igb2: Using MSIX interrupts with 5 vectors igb2: allocated for 4 tx_queues igb2: allocated for 4 rx_queues igb2: Ethernet address: 00:25:90:09:c8:da igb2: netmap queues/slots: TX 4/1024, RX 4/1024 igb3: port 0xe880-0xe89f mem 0xfbd60000-0xfbd7ffff,0xfbd40000-0xfbd5ffff,0xfbd1c000-0xfbd1ffff irq 25 at device 0.1 on pci3 igb3: attach_pre capping queues at 8 igb3: using 1024 tx descriptors and 1024 rx descriptors igb3: msix_init qsets capped at 8 igb3: pxm cpus: 4 queue msgs: 9 admincnt: 1 igb3: using 4 rx queues ¿€¹igb3: Using MSIX interrupts with 5 vectors igb3: allocated for 4 tx_queues igb3: allocated for 4 rx_queues igb3: Ethernet address: 00:25:90:09:c8:db igb3: netmap queues/slots: TX 4/1024, RX 4/1024 pcib4: at device 7.0 on pci0 pci4: on pcib4 pcib5: at device 9.0 on pci0 pci5: on pcib5 pci0: at device 20.0 (no driver attached) pci0: at device 20.1 (no driver attached) pci0: at device 20.2 (no driver attached) pci0: at device 20.3 (no driver attached) uhci0: port 0xcc00-0xcc1f irq 16 at device 26.0 on pci0 (XEN) d0: bind: m_gsi=16 g_gsi=16 uhci0: LegSup = 0x2f00 usbus0 on uhci0 usbus0: 12Mbps Full Speed USB v1.0 uhci1: port 0xc880-0xc89f irq 21 at device 26.1 on pci0 (XEN) d0: bind: m_gsi=21 g_gsi=21 uhci1: LegSup = 0x2f00 usbus1 on uhci1 usbus1: 12Mbps Full Speed USB v1.0 uhci2: port 0xc800-0xc81f irq 19 at device 26.2 on pci0 (XEN) d0: bind: m_gsi=19 g_gsi=19 uhci2: LegSup = 0x2f00 usbus2 on uhci2 usbus2: 12Mbps Full Speed USB v1.0 ehci0: mem 0xfbeda000-0xfbeda3ff irq 18 at device 26.7 on pci0 (XEN) d0: bind: m_gsi=18 g_gsi=18 usbus3: EHCI version 1.0 usbus3 on ehci0 usbus3: 480Mbps High Speed USB v2.0 uhci3: port 0xc480-0xc49f irq 23 at device 29.0 on pci0 (XEN) d0: bind: m_gsi=23 g_gsi=23 uhci3: LegSup = 0x2f00 usbus4 on uhci3 usbus4: 12Mbps Full Speed USB v1.0 uhci4: port 0xc400-0xc41f irq 19 at device 29.1 on pci0 uhci4: LegSup = 0x2f00 usbus5 on uhci4 usbus5: 12Mbps Full Speed USB v1.0 uhci5: port 0xc080-0xc09f irq 18 at device 29.2 on pci0 uhci5: LegSup = 0x2f00 usbus6 on uhci5 usbus6: 12Mbps Full Speed USB v1.0 ehci1: mem 0xfbed8000-0xfbed83ff irq 23 at device 29.7 on pci0 usbus7: EHCI version 1.0 usbus7 on ehci1 usbus7: 480Mbps High Speed USB v2.0 pcib6: at device 30.0 on pci0 pci6: on pcib6 vgapci0: mem 0xf9000000-0xf9ffffff,0xfaffc000-0xfaffffff,0xfb000000-0xfb7fffff irq 18 at device 1.0 on pci6 vgapci0: Boot video device isab0: at device 31.0 on pci0 isa0: on isab0 ahci0: port 0xb480-0xb487,0xc000-0xc003,0xbc00-0xbc07,0xb880-0xb883,0xb800-0xb81f mem 0xfbed6000-0xfbed67ff irq 19 at device 31.2 on pci0 ahci0: AHCI v1.20 with 6 3Gbps ports, Port Multiplier not supported ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 ahcich2: at channel 2 on ahci0 ahcich3: at channel 3 on ahci0 ahcich4: at channel 4 on ahci0 ahcich5: at channel 5 on ahci0 ahciem0: on ahci0 acpi_button0: on acpi0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 (XEN) d0: bind: m_gsi=4 g_gsi=4 xenpv0: on motherboard granttable0: on xenpv0 xen_et0: on xenpv0 Event timer "XENTIMER" frequency 1000000000 Hz quality 950 Timecounter "XENTIMER" frequency 1000000000 Hz quality 950 xen_et0: registered as a time-of-day clock, resolution 0.000001s xenstore0: on xenpv0 xsd_dev0: on xenpv0 evtchn0: on xenpv0 privcmd0: on xenpv0 gntdev0: on xenpv0 debug0: on xenpv0 xc0: on xenpv0 ipmi0: on isa0 ipmi0: KCS mode found at io 0xca2 alignment 0x1 on isa orm0: at iomem 0xc0000-0xc7fff,0xc8000-0xc8fff,0xc9000-0xc9fff,0xca000-0xcafff pnpid ORM0000 on isa0 vga0: at port 0x3d0-0x3db iomem 0xb8000-0xbffff pnpid PNP0900 on isa0 atkbdc0: at port 0x60,0x64 on isa0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 (XEN) d0: bind: m_gsi=1 g_gsi=1 atkbd0: [GIANT-LOCKED] atkbdc0: non-PNP ISA device will be removed from GENERIC in FreeBSD 12. Timecounters tick every 10.000 msec ipmi0: IPMI device rev. 1, firmware rev. 3.13, version 2.0, device support mask 0xbf ipmi0: Number of channels 2 ipmi0: Attached watchdog ipmi0: Establishing power cycle handler ugen5.1: at usbus5 ugen4.1: at usbus4 uhub0: on usbus5 ugen6.1: at usbus6 uhub1: ugen7.1: at usbus7 on usbus4 uhub2: on usbus7 ugen3.1: at usbus3 uhub3: ugen0.1: at usbus0 on usbus6 uhub4: on usbus3 uhub5: on usbus0 ugen2.1: at usbus2 ugen1.1: at usbus1 uhub6: on usbus2 uhub7: on usbus1 uhub0: 2 ports with 2 removable, self powered uhub1: 2 ports with 2 removable, self powered uhub3: 2 ports with 2 removable, self powered uhub5: 2 ports with 2 removable, self powered uhub7: 2 ports with 2 removable, self powered uhub6: 2 ports with 2 removable, self powered ses0 at ahciem0 bus 0 scbus6 target 0 lun 0 ses0: SEMB S-E-S 2.00 device ses0: SEMB SES Device ada0 at ahcich0 bus 0 scbus0 target 0 lun 0 ada0: ACS-3 ATA SATA 3.x device ada0: Serial Number ZGY0KSAS ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada0: Command Queueing enabled ada0: 3815447MB (7814037168 512 byte sectors) ada0: quirks=0x1<4K> ada1 at ahcich1 bus 0 scbus1 target 0 lun 0 ada1: ACS-3 ATA SATA 3.x device ada1: Serial Number ZGY0KSC9 ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada1: Command Queueing enabled ada1: 3815447MB (7814037168 512 byte sectors) ada1: quirks=0x1<4K> ada2 at ahcich2 bus 0 scbus2 target 0 lun 0 ada2: ACS-3 ATA SATA 3.x device ada2: Serial Number ZGY0KS9B ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada2: Command Queueing enabled ada2: 3815447MB (7814037168 512 byte sectors) ada2: quirks=0x1<4K> ada3 at ahcich3 bus 0 scbus3 target 0 lun 0 ada3: ACS-3 ATA SATA 3.x device ada3: Serial Number ZDH1SGEB ada3: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada3: Command Queueing enabled ada3: 3815447MB (7814037168 512 byte sectors) ada3: quirks=0x1<4K> Trying to mount root from ufs:/dev/gpt/gptroot [rw]... uhub4: 6 ports with 6 removable, self powered uhub2: 6 ports with 6 removable, self powered cd0 at ahcich4 bus 0 scbus4 target 0 lun 0 cd0: Removable CD-ROM SCSI device cd0: 150.000MB/s transfers (SATA 1.x, UDMA2, ATAPI 12bytes, PIO 8192bytes) cd0: Attempt to query device size failed: NOT READY, Medium not present - tray closed GWARNING: /mnt was not properly dismounted ugen1.2: at usbus1 ukbd0 on uhub7 ukbd0: on usbus1 kbd2 at ukbd0 ahcich0: Timeout on slot 15 port 0 ahcich0: is 00000008 cs 00000000 ss 00000000 rs 0000b000 tfd 40 serr 00000000 cmd 0004cf17 (ada0:ahcich0:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 c0 78 08 40 13 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: Command timeout (ada0:ahcich0:0:0:0): Retrying command, 3 more tries remain ahcich0: Timeout on slot 16 port 0 ahcich0: is 00000002 cs 00000000 ss 00000000 rs 00010000 tfd 50 serr 00000000 cmd 0004d017 (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: Command timeout (aprobe0:ahcich0:0:0:0): Retrying command, 0 more tries remain ahcich0: Timeout on slot 17 port 0 ahcich0: is 00000002 cs 00000000 ss 00000000 rs 00020000 tfd 50 serr 00000000 cmd 0004d117 (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: Command timeout (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted ahcich0: Timeout on slot 18 port 0 ahcich0: is 00000002 cs 00000000 ss 00000000 rs 00040000 tfd 50 serr 00000000 cmd 0004d217 (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: Command timeout (aprobe0:ahcich0:0:0:0): Error 5, Retry was blocked ada0 at ahcich0 bus 0 scbus0 target 0 lun 0 ada0: s/n ZGY0KSAS detached ahcich0: Timeout on slot 19 port 0 ahcich0: is 00000002 cs 00000000 ss 00000000 rs 00080000 tfd 50 serr 00000000 cmd 0004d317 (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: Command timeout (aprobe0:ahcich0:0:0:0): Retrying command, 0 more tries remain ahcich0: Timeout on slot 20 port 0 ahcich0: is 00000002 cs 00000000 ss 00000000 rs 00100000 tfd 50 serr 00000000 cmd 0004d417 (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: Command timeout (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted ahcich0: Timeout on slot 25 port 0 ahcich0: is 00000002 cs 00000000 ss 00000000 rs 02000000 tfd 50 serr 00000000 cmd 0004d917 (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: Command timeout (aprobe0:ahcich0:0:0:0): Retrying command, 0 more tries remain (XEN) *** Serial input to Xen (type 'CTRL-a' three times to switch input) ahcich0: Timeout on slot 26 port 0 ahcich0: is 00000002 cs 00000000 ss 00000000 rs 04000000 tfd 50 serr 00000000 cmd 0004da17 (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: Command timeout (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted (XEN) IRQ information: (XEN) IRQ: 0 affinity:000001 vec:f0 type=IO-APIC-edge status=00000000 time.c#timer_interrupt() (XEN) IRQ: 1 affinity:001000 vec:55 type=IO-APIC-edge status=00000014 in-flight=0 domain-list=0: 1(-M-), (XEN) IRQ: 3 affinity:00ffff vec:f1 type=IO-APIC-edge status=00000000 ns16550.c#ns16550_interrupt() (XEN) IRQ: 4 affinity:001000 vec:4d type=IO-APIC-edge status=00000014 in-flight=0 domain-list=0: 4(-M-), (XEN) IRQ: 5 affinity:000001 vec:40 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 6 affinity:000001 vec:48 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 7 affinity:000001 vec:50 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 8 affinity:001000 vec:a8 type=IO-APIC-edge status=00000010 in-flight=0 domain-list=0: 8(-M-), (XEN) IRQ: 9 affinity:001000 vec:a0 type=IO-APIC-level status=00000010 in-flight=0 domain-list=0: 9(-M-), (XEN) IRQ: 10 affinity:000001 vec:68 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 11 affinity:000001 vec:70 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 12 affinity:000001 vec:78 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 13 affinity:000001 vec:88 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 14 affinity:000001 vec:90 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 15 affinity:000001 vec:98 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 16 affinity:001000 vec:83 type=IO-APIC-level status=00000010 in-flight=0 domain-list=0: 16(-M-), (XEN) IRQ: 18 affinity:001000 vec:b3 type=IO-APIC-level status=00000010 in-flight=0 domain-list=0: 18(-M-), (XEN) IRQ: 19 affinity:001000 vec:a3 type=IO-APIC-level status=00000010 in-flight=0 domain-list=0: 19(-M-), (XEN) IRQ: 21 affinity:001000 vec:93 type=IO-APIC-level status=00000010 in-flight=0 domain-list=0: 21(-M-), (XEN) IRQ: 23 affinity:001000 vec:c3 type=IO-APIC-level status=00000010 in-flight=0 domain-list=0: 23(-M-), (XEN) IRQ: 48 affinity:00ffff vec:28 type=DMA_MSI status=00000000 iommu.c#iommu_page_fault() (XEN) IRQ: 49 affinity:001000 vec:c8 type=PCI-MSI/-X status=00000010 in-flight=0 domain-list=0:815(-M-), (XEN) IRQ: 50 affinity:001000 vec:29 type=PCI-MSI/-X status=00000030 in-flight=0 domain-list=0:814(-M-), (XEN) IRQ: 51 affinity:001000 vec:49 type=PCI-MSI/-X status=00000030 in-flight=0 domain-list=0:813(-M-), (XEN) IRQ: 52 affinity:001000 vec:69 type=PCI-MSI/-X status=00000030 in-flight=0 domain-list=0:812(-M-), (XEN) IRQ: 53 affinity:001000 vec:79 type=PCI-MSI/-X status=00000030 in-flight=0 domain-list=0:811(-M-), (XEN) IRQ: 54 affinity:001000 vec:89 type=PCI-MSI/-X status=00000010 in-flight=0 domain-list=0:810(-M-), (XEN) IRQ: 55 affinity:001000 vec:99 type=PCI-MSI/-X status=00000030 in-flight=0 domain-list=0:809(-M-), (XEN) IRQ: 56 affinity:001000 vec:a9 type=PCI-MSI/-X status=00000030 in-flight=0 domain-list=0:808(-M-), (XEN) IRQ: 57 affinity:001000 vec:b9 type=PCI-MSI/-X status=00000030 in-flight=0 domain-list=0:807(-M-), (XEN) IRQ: 58 affinity:001000 vec:c9 type=PCI-MSI/-X status=00000030 in-flight=0 domain-list=0:806(-M-), (XEN) IRQ: 59 affinity:001000 vec:2a type=PCI-MSI/-X status=00000030 in-flight=0 domain-list=0:805(-M-), (XEN) IRQ: 60 affinity:001000 vec:4a type=PCI-MSI/-X status=00000030 in-flight=0 domain-list=0:804(-M-), (XEN) IRQ: 61 affinity:001000 vec:6a type=PCI-MSI/-X status=00000030 in-flight=0 domain-list=0:803(-M-), (XEN) IRQ: 62 affinity:001000 vec:92 type=PCI-MSI/-X status=00000030 in-flight=0 domain-list=0:802(-M-), (XEN) IRQ: 63 affinity:001000 vec:a2 type=PCI-MSI/-X status=00000030 in-flight=0 domain-list=0:801(-M-), (XEN) IRQ: 64 affinity:001000 vec:c2 type=PCI-MSI/-X status=00000030 in-flight=0 domain-list=0:800(-M-), (XEN) IRQ: 65 affinity:001000 vec:23 type=PCI-MSI/-X status=00000030 in-flight=0 domain-list=0:799(-M-), (XEN) IRQ: 66 affinity:001000 vec:43 type=PCI-MSI/-X status=00000030 in-flight=0 domain-list=0:798(-M-), (XEN) IRQ: 67 affinity:001000 vec:63 type=PCI-MSI/-X status=00000030 in-flight=0 domain-list=0:797(-M-), (XEN) IRQ: 68 affinity:001000 vec:73 type=PCI-MSI/-X status=00000030 in-flight=0 domain-list=0:796(-M-), (XEN) IRQ: 69 affinity:001000 vec:8c type=PCI-MSI status=00000010 in-flight=0 domain-list=0:780(-M-), (XEN) IRQ: 70 affinity:001000 vec:94 type=PCI-MSI status=00000010 in-flight=0 domain-list=0:781(-M-), (XEN) IRQ: 71 affinity:001000 vec:9c type=PCI-MSI status=00000010 in-flight=0 domain-list=0:782(-M-), (XEN) IRQ: 72 affinity:001000 vec:a4 type=PCI-MSI status=00000010 in-flight=0 domain-list=0:783(-M-), (XEN) IRQ: 73 affinity:001000 vec:ac type=PCI-MSI status=00000010 in-flight=0 domain-list=0:784(-M-), (XEN) IRQ: 74 affinity:001000 vec:b4 type=PCI-MSI status=00000030 in-flight=0 domain-list=0:785(-M-), (XEN) IRQ: 75 affinity:001000 vec:bc type=PCI-MSI status=00000030 in-flight=0 domain-list=0:786(-M-), (XEN) IRQ: 76 affinity:001000 vec:c4 type=PCI-MSI status=00000030 in-flight=0 domain-list=0:787(-M-), (XEN) IRQ: 77 affinity:001000 vec:cc type=PCI-MSI status=00000030 in-flight=0 domain-list=0:788(-M-), (XEN) IRQ: 78 affinity:001000 vec:d4 type=PCI-MSI status=00000030 in-flight=0 domain-list=0:789(-M-), (XEN) IRQ: 79 affinity:001000 vec:dc type=PCI-MSI status=00000030 in-flight=0 domain-list=0:790(-M-), (XEN) IRQ: 80 affinity:001000 vec:25 type=PCI-MSI status=00000030 in-flight=0 domain-list=0:791(-M-), (XEN) IRQ: 81 affinity:001000 vec:2d type=PCI-MSI status=00000030 in-flight=0 domain-list=0:792(-M-), (XEN) IRQ: 82 affinity:001000 vec:35 type=PCI-MSI status=00000030 in-flight=0 domain-list=0:793(-M-), (XEN) IRQ: 83 affinity:001000 vec:3d type=PCI-MSI status=00000030 in-flight=0 domain-list=0:794(-M-), (XEN) IRQ: 84 affinity:001000 vec:45 type=PCI-MSI status=00000030 in-flight=0 domain-list=0:795(-M-), (XEN) Direct vector information: (XEN) 0x20 -> vmac.c#irq_move_cleanup_interrupt() (XEN) 0xf2 -> mce_intel.c#cmci_interrupt() (XEN) 0xf3 -> mce_intel.c#intel_thermal_interrupt() (XEN) 0xf9 -> vmac.c#pmu_apic_interrupt() (XEN) 0xfa -> vmac.c#apic_timer_interrupt() (XEN) 0xfb -> vmac.c#call_function_interrupt() (XEN) 0xfc -> vmac.c#event_check_interrupt() (XEN) 0xfd -> vmac.c#invalidate_interrupt() (XEN) 0xfe -> vmac.c#error_interrupt() (XEN) 0xff -> vmac.c#spurious_interrupt() (XEN) IO-APIC interrupt information: (XEN) IRQ 0 Vec240: (XEN) Apic 0x00, Pin 2: vec=f0 delivery=Fixed dest=P status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 1 Vec 85: (XEN) Apic 0x00, Pin 1: vec=55 delivery=Fixed dest=P status=0 polarity=0 irr=0 trig=E mask=0 dest_id:20 (XEN) IRQ 3 Vec241: (XEN) Apic 0x00, Pin 3: vec=f1 delivery=Fixed dest=P status=0 polarity=0 irr=0 trig=E mask=0 dest_id:5 (XEN) IRQ 4 Vec 77: (XEN) Apic 0x00, Pin 4: vec=4d delivery=Fixed dest=P status=0 polarity=0 irr=0 trig=E mask=0 dest_id:20 (XEN) IRQ 5 Vec 64: (XEN) Apic 0x00, Pin 5: vec=40 delivery=Fixed dest=P status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 6 Vec 72: (XEN) Apic 0x00, Pin 6: vec=48 delivery=Fixed dest=P status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 7 Vec 80: (XEN) Apic 0x00, Pin 7: vec=50 delivery=Fixed dest=P status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 8 Vec168: (XEN) Apic 0x00, Pin 8: vec=a8 delivery=Fixed dest=P status=0 polarity=0 irr=0 trig=E mask=0 dest_id:20 (XEN) IRQ 9 Vec160: (XEN) Apic 0x00, Pin 9: vec=a0 delivery=Fixed dest=P status=0 polarity=0 irr=0 trig=L mask=0 dest_id:20 (XEN) IRQ 10 Vec104: (XEN) Apic 0x00, Pin 10: vec=68 delivery=Fixed dest=P status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 11 Vec112: (XEN) Apic 0x00, Pin 11: vec=70 delivery=Fixed dest=P status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 12 Vec120: (XEN) Apic 0x00, Pin 12: vec=78 delivery=Fixed dest=P status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 13 Vec136: (XEN) Apic 0x00, Pin 13: vec=88 delivery=Fixed dest=P status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 14 Vec144: (XEN) Apic 0x00, Pin 14: vec=90 delivery=Fixed dest=P status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 15 Vec152: (XEN) Apic 0x00, Pin 15: vec=98 delivery=Fixed dest=P status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 16 Vec131: (XEN) Apic 0x00, Pin 16: vec=83 delivery=Fixed dest=P status=0 polarity=1 irr=0 trig=L mask=0 dest_id:20 (XEN) IRQ 18 Vec179: (XEN) Apic 0x00, Pin 18: vec=b3 delivery=Fixed dest=P status=0 polarity=1 irr=0 trig=L mask=0 dest_id:20 (XEN) IRQ 19 Vec163: (XEN) Apic 0x00, Pin 19: vec=a3 delivery=Fixed dest=P status=0 polarity=1 irr=0 trig=L mask=0 dest_id:20 (XEN) IRQ 21 Vec147: (XEN) Apic 0x00, Pin 21: vec=93 delivery=Fixed dest=P status=0 polarity=1 irr=1 trig=L mask=0 dest_id:20 (XEN) IRQ 23 Vec195: (XEN) Apic 0x00, Pin 23: vec=c3 delivery=Fixed dest=P status=0 polarity=1 irr=0 trig=L mask=0 dest_id:20 (XEN) Physical memory information: (XEN) Xen heap: 0kB free (XEN) heap[09]: 1024kB free (XEN) heap[10]: 2048kB free (XEN) heap[11]: 4096kB free (XEN) heap[12]: 8192kB free (XEN) heap[13]: 16384kB free (XEN) heap[14]: 32768kB free (XEN) heap[15]: 65536kB free (XEN) heap[16]: 131072kB free (XEN) heap[17]: 262144kB free (XEN) heap[18]: 524288kB free (XEN) heap[19]: 1048576kB free (XEN) DMA heap: 2096128kB free (XEN) heap[20]: 1036336kB free (XEN) heap[21]: 4194304kB free (XEN) heap[22]: 8388608kB free (XEN) heap[23]: 16777216kB free (XEN) heap[24]: 30933460kB free (XEN) heap[25]: 31897220kB free (XEN) Dom heap: 93227144kB free (XEN) number of MP IRQ sources: 15. (XEN) number of IO-APIC #8 registers: 24. (XEN) number of IO-APIC #9 registers: 24. (XEN) testing the IO APIC....................... (XEN) IO APIC #8...... (XEN) .... register #00: 08000000 (XEN) ....... : physical APIC id: 08 (XEN) ....... : Delivery Type: 0 (XEN) ....... : LTS : 0 (XEN) .... register #01: 00170020 (XEN) ....... : max redirection entries: 0017 (XEN) ....... : PRQ implemented: 0 (XEN) ....... : IO APIC version: 0020 (XEN) .... IRQ redirection table: (XEN) NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect: (XEN) 00 000 00 1 0 0 0 0 0 0 00 (XEN) 01 014 04 0 0 0 0 0 0 0 55 (XEN) 02 000 00 0 0 0 0 0 0 0 F0 (XEN) 03 005 05 0 0 0 0 0 0 0 F1 (XEN) 04 014 04 0 0 0 0 0 0 0 4D (XEN) 05 000 00 0 0 0 0 0 0 0 40 (XEN) 06 000 00 0 0 0 0 0 0 0 48 (XEN) 07 000 00 0 0 0 0 0 0 0 50 (XEN) 08 014 04 0 0 0 0 0 0 0 A8 (XEN) 09 014 04 0 1 0 0 0 0 0 A0 (XEN) 0a 000 00 0 0 0 0 0 0 0 68 (XEN) 0b 000 00 0 0 0 0 0 0 0 70 (XEN) 0c 000 00 0 0 0 0 0 0 0 78 (XEN) 0d 000 00 0 0 0 0 0 0 0 88 (XEN) 0e 000 00 0 0 0 0 0 0 0 90 (XEN) 0f 000 00 0 0 0 0 0 0 0 98 (XEN) 10 014 04 0 1 0 1 0 0 0 83 (XEN) 11 000 00 1 0 0 0 0 0 0 00 (XEN) 12 014 04 0 1 0 1 0 0 0 B3 (XEN) 13 014 04 0 1 0 1 0 0 0 A3 (XEN) 14 000 00 1 0 0 0 0 0 0 00 (XEN) 15 014 04 0 1 1 1 0 0 0 93 (XEN) 16 000 00 1 0 0 0 0 0 0 00 (XEN) 17 014 04 0 1 0 1 0 0 0 C3 (XEN) IO APIC #9...... (XEN) .... register #00: 09000000 (XEN) ....... : physical APIC id: 09 (XEN) ....... : Delivery Type: 0 (XEN) ....... : LTS : 0 (XEN) .... register #01: 00170020 (XEN) ....... : max redirection entries: 0017 (XEN) ....... : PRQ implemented: 0 (XEN) ....... : IO APIC version: 0020 (XEN) .... register #02: 00000000 (XEN) ....... : arbitration: 00 (XEN) .... register #03: 00000001 (XEN) ....... : Boot DT : 1 (XEN) .... IRQ redirection table: (XEN) NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect: (XEN) 00 000 00 1 0 0 0 0 0 0 00 (XEN) 01 000 00 1 0 0 0 0 0 0 00 (XEN) 02 000 00 1 0 0 0 0 0 0 00 (XEN) 03 000 00 1 0 0 0 0 0 0 00 (XEN) 04 000 00 1 0 0 0 0 0 0 00 (XEN) 05 000 00 1 0 0 0 0 0 0 00 (XEN) 06 000 00 1 0 0 0 0 0 0 00 (XEN) 07 000 00 1 0 0 0 0 0 0 00 (XEN) 08 000 00 1 0 0 0 0 0 0 00 (XEN) 09 000 00 1 0 0 0 0 0 0 00 (XEN) 0a 000 00 1 0 0 0 0 0 0 00 (XEN) 0b 000 00 1 0 0 0 0 0 0 00 (XEN) 0c 000 00 1 0 0 0 0 0 0 00 (XEN) 0d 000 00 1 0 0 0 0 0 0 00 (XEN) 0e 000 00 1 0 0 0 0 0 0 00 (XEN) 0f 000 00 1 0 0 0 0 0 0 00 (XEN) 10 000 00 1 0 0 0 0 0 0 00 (XEN) 11 000 00 1 0 0 0 0 0 0 00 (XEN) 12 000 00 1 0 0 0 0 0 0 00 (XEN) 13 000 00 1 0 0 0 0 0 0 00 (XEN) 14 000 00 1 0 0 0 0 0 0 00 (XEN) 15 000 00 1 0 0 0 0 0 0 00 (XEN) 16 000 00 1 0 0 0 0 0 0 00 (XEN) 17 000 00 1 0 0 0 0 0 0 00 (XEN) Using vector-based indexing (XEN) IRQ to pin mappings: (XEN) IRQ240 -> 0:2 (XEN) IRQ85 -> 0:1 (XEN) IRQ241 -> 0:3 (XEN) IRQ77 -> 0:4 (XEN) IRQ64 -> 0:5 (XEN) IRQ72 -> 0:6 (XEN) IRQ80 -> 0:7 (XEN) IRQ168 -> 0:8 (XEN) IRQ160 -> 0:9 (XEN) IRQ104 -> 0:10 (XEN) IRQ112 -> 0:11 (XEN) IRQ120 -> 0:12 (XEN) IRQ136 -> 0:13 (XEN) IRQ144 -> 0:14 (XEN) IRQ152 -> 0:15 (XEN) IRQ131 -> 0:16 (XEN) IRQ179 -> 0:18 (XEN) IRQ163 -> 0:19 (XEN) IRQ147 -> 0:21 (XEN) IRQ195 -> 0:23 (XEN) .................................... done. ahcich0: Timeout on slot 29 port 0 ahcich0: is 00000002 cs 00000000 ss 00000000 rs 20000000 ¿€¹(aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: Command timeout (aprobe0:ahcich0:0:0:0): Retrying command, 0 more tries remain (XEN) number of MP IRQ sources: 15. (XEN) number of IO-APIC #8 registers: 24. (XEN) number of IO-APIC #9 registers: 24. (XEN) testing the IO APIC....................... (XEN) IO APIC #8...... (XEN) .... register #00: 08000000 (XEN) ....... : physical APIC id: 08 (XEN) ....... : Delivery Type: 0 (XEN) ....... : LTS : 0 (XEN) .... register #01: 00170020 (XEN) ....... : max redirection entries: 0017 (XEN) ....... : PRQ implemented: 0 (XEN) ....... : IO APIC version: 0020 (XEN) .... IRQ redirection table: (XEN) NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect: (XEN) 00 000 00 1 0 0 0 0 0 0 00 (XEN) 01 014 04 0 0 0 0 0 0 0 55 (XEN) 02 000 00 0 0 0 0 0 0 0 F0 (XEN) 03 005 05 0 0 0 0 0 0 0 F1 (XEN) 04 014 04 0 0 0 0 0 0 0 4D (XEN) 05 000 00 0 0 0 0 0 0 0 40 (XEN) 06 000 00 0 0 0 0 0 0 0 48 (XEN) 07 000 00 0 0 0 0 0 0 0 50 (XEN) 08 014 04 0 0 0 0 0 0 0 A8 (XEN) 09 014 04 0 1 0 0 0 0 0 A0 (XEN) 0a 000 00 0 0 0 0 0 0 0 68 (XEN) 0b 000 00 0 0 0 0 0 0 0 70 (XEN) 0c 000 00 0 0 0 0 0 0 0 78 (XEN) 0d 000 00 0 0 0 0 0 0 0 88 (XEN) 0e 000 00 0 0 0 0 0 0 0 90 (XEN) 0f 000 00 0 0 0 0 0 0 0 98 (XEN) 10 014 04 0 1 0 1 0 0 0 83 (XEN) 11 000 00 1 0 0 0 0 0 0 00 (XEN) 12 014 04 0 1 0 1 0 0 0 B3 (XEN) 13 014 04 0 1 0 1 0 0 0 A3 (XEN) 14 000 00 1 0 0 0 0 0 0 00 (XEN) 15 014 04 0 1 1 1 0 0 0 93 (XEN) 16 000 00 1 0 0 0 0 0 0 00 (XEN) 17 014 04 0 1 0 1 0 0 0 C3 (XEN) IO APIC #9...... (XEN) .... register #00: 09000000 (XEN) ....... : physical APIC id: 09 (XEN) ....... : Delivery Type: 0 (XEN) ....... : LTS : 0 (XEN) .... register #01: 00170020 (XEN) ....... : max redirection entries: 0017 (XEN) ....... : PRQ implemented: 0 (XEN) ....... : IO APIC version: 0020 (XEN) .... register #02: 00000000 (XEN) ....... : arbitration: 00 (XEN) .... register #03: 00000001 (XEN) ....... : Boot DT : 1 (XEN) .... IRQ redirection table: (XEN) NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect: (XEN) 00 000 00 1 0 0 0 0 0 0 00 (XEN) 01 000 00 1 0 0 0 0 0 0 00 (XEN) 02 000 00 1 0 0 0 0 0 0 00 (XEN) 03 000 00 1 0 0 0 0 0 0 00 (XEN) 04 000 00 1 0 0 0 0 0 0 00 (XEN) 05 000 00 1 0 0 0 0 0 0 00 (XEN) 06 000 00 1 0 0 0 0 0 0 00 (XEN) 07 000 00 1 0 0 0 0 0 0 00 (XEN) 08 000 00 1 0 0 0 0 0 0 00 (XEN) 09 000 00 1 0 0 0 0 0 0 00 (XEN) 0a 000 00 1 0 0 0 0 0 0 00 (XEN) 0b 000 00 1 0 0 0 0 0 0 00 (XEN) 0c 000 00 1 0 0 0 0 0 0 00 (XEN) 0d 000 00 1 0 0 0 0 0 0 00 (XEN) 0e 000 00 1 0 0 0 0 0 0 00 (XEN) 0f 000 00 1 0 0 0 0 0 0 00 (XEN) 10 000 00 1 0 0 0 0 0 0 00 (XEN) 11 000 00 1 0 0 0 0 0 0 00 (XEN) 12 000 00 1 0 0 0 0 0 0 00 (XEN) 13 000 00 1 0 0 0 0 0 0 00 (XEN) 14 000 00 1 0 0 0 0 0 0 00 (XEN) 15 000 00 1 0 0 0 0 0 0 00 (XEN) 16 000 00 1 0 0 0 0 0 0 00 (XEN) 17 000 00 1 0 0 0 0 0 0 00 (XEN) Using vector-based indexing (XEN) IRQ to pin mappings: (XEN) IRQ240 -> 0:2 (XEN) IRQ85 -> 0:1 (XEN) IRQ241 -> 0:3 (XEN) IRQ77 -> 0:4 (XEN) IRQ64 -> 0:5 (XEN) IRQ72 -> 0:6 (XEN) IRQ80 -> 0:7 (XEN) IRQ168 -> 0:8 (XEN) IRQ160 -> 0:9 (XEN) IRQ104 -> 0:10 (XEN) IRQ112 -> 0:11 (XEN) IRQ120 -> 0:12 (XEN) IRQ136 -> 0:13 (XEN) IRQ144 -> 0:14 (XEN) IRQ152 -> 0:15 (XEN) IRQ131 -> 0:16 (XEN) IRQ179 -> 0:18 (XEN) IRQ163 -> 0:19 (XEN) IRQ147 -> 0:21 (XEN) IRQ195 -> 0:23 (XEN) .................................... done. From owner-freebsd-xen@freebsd.org Sat Oct 12 09:59:17 2019 Return-Path: Delivered-To: freebsd-xen@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8723A1511D0 for ; Sat, 12 Oct 2019 09:59:17 +0000 (UTC) (envelope-from roger.pau@citrix.com) Received: from esa5.hc3370-68.iphmx.com (esa5.hc3370-68.iphmx.com [216.71.155.168]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "esa5.hc3370-68.iphmx.com", Issuer "HydrantID SSL ICA G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46r0fD45Rwz3KRr for ; Sat, 12 Oct 2019 09:59:16 +0000 (UTC) (envelope-from roger.pau@citrix.com) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1570874356; h=date:from:to:cc:subject:message-id:references: mime-version:in-reply-to; bh=UFc+7YPHpd1zxAhgZeQeFX4GE5UfJKgjapHlim/ifs0=; b=dXfgK71lcgxabKENd/JMvt5gB/Jgy7WlnxNT6qA4gvqNefUHLuVkk/c1 LdMxSrSCufsK0JNbaePYQfzG8oLrT3Nshg19Xp6fX07sZSEcpu6qUNRKg S91IYRPLG5EU6DCFvGW4nzZw2dJPJA00QpEEQn2QpV4nTfSMjZG6jJrf+ I=; Received-SPF: None (esa5.hc3370-68.iphmx.com: no sender authenticity information available from domain of roger.pau@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa5.hc3370-68.iphmx.com: domain of roger.pau@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa5.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: yNrHCn9zfrwenx56CAZ7Ox9u44yPnnazC5zBTpXoaYCaqnSJICcPPyQtJ3RRCVTsf/cfaKs6vv n+CgVhgC35rW6u78fLLlrnUb9LLugTA/5PbeFKW/lEuYL2zt7H3c+HctJwrnuNKxtJaaDEgqI/ ZDgLqtegj6OHo321POYueB5VH1IDyhuOLo3TgDPUcbvnQwooeCFqFAueQzX+EpMTr56AVkZTrk 5j/YQSvs1yP3f4OEglMdxNYFkQojMo/27+yy/vVCQB6Gm30teKHuhado3VIWU3HhrHHDxw2FBE Ek8= X-SBRS: 2.7 X-MesageID: 7074688 X-Ironport-Server: esa5.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.67,287,1566878400"; d="scan'208";a="7074688" Date: Sat, 12 Oct 2019 11:59:00 +0200 From: Roger Pau =?iso-8859-1?Q?Monn=E9?= To: Brian Buhrow CC: Subject: Re: i/o is very slow on FreeBSD dom0 with Xen-4.12 and Freebsd-12 Message-ID: <20191012095900.GA14005@Air-de-Roger> References: <201910112153.x9BLrQSI028763@lothlorien.nfbcal.org> <201910112322.x9BNMWlp020596@lothlorien.nfbcal.org> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: <201910112322.x9BNMWlp020596@lothlorien.nfbcal.org> User-Agent: Mutt/1.12.2 (2019-09-21) X-ClientProxiedBy: AMSPEX02CAS01.citrite.net (10.69.22.112) To AMSPEX02CL02.citrite.net (10.69.22.126) X-Rspamd-Queue-Id: 46r0fD45Rwz3KRr X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=citrix.com header.s=securemail header.b=dXfgK71l; dmarc=pass (policy=none) header.from=citrix.com; spf=pass (mx1.freebsd.org: domain of roger.pau@citrix.com designates 216.71.155.168 as permitted sender) smtp.mailfrom=roger.pau@citrix.com X-Spamd-Result: default: False [-6.00 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[citrix.com:s=securemail]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; RCVD_DKIM_ARC_DNSWL_MED(-0.50)[]; DWL_DNSWL_LOW(-1.00)[citrix.com.dwl.dnswl.org : 127.0.9.1]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_MED(-0.20)[168.155.71.216.list.dnswl.org : 127.0.3.2]; DKIM_TRACE(0.00)[citrix.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[citrix.com,none]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MID_RHS_NOT_FQDN(0.50)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:16417, ipnet:216.71.154.0/23, country:US]; IP_SCORE(-1.80)[ip: (-5.06), ipnet: 216.71.154.0/23(-2.60), asn: 16417(-1.27), country: US(-0.05)] X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Oct 2019 09:59:17 -0000 On Fri, Oct 11, 2019 at 04:22:32PM -0700, Brian Buhrow wrote: > hello. Following up on this issue yet again, I'm pretty sure I have > the hardware discussed in the link below. I tried disabling interrupt > remapping as the article suggests, but that just makes matters worse. As I > understand it, when FreeBSD is running as a dom0, it must have iommu > interrupt remapping enabled in order to function. PVH dom0 requires iommu DMA remapping. Interrupt remapping is not mandated for PVH dom0. Can you also post the log when booting with iommu=verbose,debug,no-intremap? Can you also try booting without the no-intremap option, and then switch to the hypervisor console (Ctrl-A on the serial line) and paste the output of the i the M and the z debug keys at the point where FreeBSD freezes? (together with the full serial log) > Should I conclude that > this hardware just cannot run FreeBSD as a dom0 or is there another work > around that I might use? I need to look deeper, I'm quite sure the issue is with Xen. There are also some bugfixes for the iommu in Xen unstable, which I can try to backport to the FreeBSD 4.12.1 Xen packages, albeit there have been a lot of changes in the iommu code, so backporting might not be feasible for some of them. I'm leaving home today and will be on PTO until Tuesday, and I don't expect to be able to look into this until then, sorry. Thanks, Roger. From owner-freebsd-xen@freebsd.org Sat Oct 12 16:43:08 2019 Return-Path: Delivered-To: freebsd-xen@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E937213BFB2 for ; Sat, 12 Oct 2019 16:43:08 +0000 (UTC) (envelope-from royger@gmail.com) Received: from mail-ot1-f41.google.com (mail-ot1-f41.google.com [209.85.210.41]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46r9cD11Z7z4J8w for ; Sat, 12 Oct 2019 16:43:07 +0000 (UTC) (envelope-from royger@gmail.com) Received: by mail-ot1-f41.google.com with SMTP id y39so10550518ota.7 for ; Sat, 12 Oct 2019 09:43:07 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=QjQSZMEqJBAK2hjLDLqjorvjDNyIqfYBfCVItUEwckI=; b=ARPhV+DKFrao03t+SMKvqU6Pp8+u2YKnAABSXehX1K322XWXNYWvQefYcnGuGdAXwM xv+amJIR6vkal+kknETcVgs5AoMIRvzpVjpGSsDDg98mUq9CfOnjwn8DGYHAp5i1glK3 i4fJgoqZn6+JveuCh5Avii4/r1qbBYYkeY/GvOqXF5s55HFd5eHnIjR7nhZX6kjUmIjA Jub4xMQvxHjHRNflk5tj+xCv4c30GOy4zya3T8aAIBQ4hHFK4uGc4NkuQhxVIzKiSG5J EkSrdMPiyq9h0zlDhnTz5ymwV+7IAgHmgZ5PwOsOHKFHxfdtb1arpFa3vxIWU/LSGtPX gYqQ== X-Gm-Message-State: APjAAAWA+naPoD9scPhA6tffi5i95kQXcd+8q8Dyg18DP4iHOpaqXuSG oJ0CaDNyc6xrtRHnB5WhhWNF2M0ovF2Ern7nESw= X-Google-Smtp-Source: APXvYqwD626pvkEInfaUhf2i79+qY17wq/XHJZoxkHbW/kZ8kwJ6OxmxQm7ZdNxxGHQ9JNgvio5wuK9prj+4PkppvRA= X-Received: by 2002:a9d:6b82:: with SMTP id b2mr16826674otq.56.1570898586231; Sat, 12 Oct 2019 09:43:06 -0700 (PDT) MIME-Version: 1.0 References: <20191012095900.GA14005@Air-de-Roger> <201910121531.x9CFVaGT014403@lothlorien.nfbcal.org> In-Reply-To: <201910121531.x9CFVaGT014403@lothlorien.nfbcal.org> From: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= Date: Sat, 12 Oct 2019 18:42:53 +0200 Message-ID: Subject: Re: i/o is very slow on FreeBSD dom0 with Xen-4.12 and Freebsd-12 To: Brian Buhrow Cc: Roger Pau Monne , freebsd-xen@freebsd.org X-Rspamd-Queue-Id: 46r9cD11Z7z4J8w X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of royger@gmail.com designates 209.85.210.41 as permitted sender) smtp.mailfrom=royger@gmail.com X-Spamd-Result: default: False [-3.19 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17:c]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-xen@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; TO_DN_SOME(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[41.210.85.209.list.dnswl.org : 127.0.5.0]; RWL_MAILSPIKE_GOOD(0.00)[41.210.85.209.rep.mailspike.net : 127.0.0.18]; IP_SCORE(-1.19)[ip: (-0.52), ipnet: 209.85.128.0/17(-3.25), asn: 15169(-2.12), country: US(-0.05)]; FORGED_SENDER(0.30)[royger@freebsd.org,royger@gmail.com]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[royger@freebsd.org,royger@gmail.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Oct 2019 16:43:09 -0000 Replying from my phone, sorry for the format. El ds., 12 d=E2=80=99oct. 2019, 17:32, Brian Buhrow va = escriure: > Hello. Here are the logs with remapping enabled, plus the i, m > and z > xen console commands. > Here's hoping you can provide a fix soon! Or, perhaps, a clue on how to > fix. > -thanks > -Brian > > > > boot_serial=3D"YES" # -h: Use serial console > comconsole_speed=3D"115200" # Set the current serial console speed > comconsole_port=3D"0x2f8" # Set the current serial console port > #console=3D"vidconsole" # A comma separated list of console(s) > console=3D"comconsole" # A comma separated list of console(s) > geom_mirror_load=3D"YES" # RAID1 disk driver (see gmirror(8)) > ahci_load=3D"YES" # ahci driver > ipmi_load=3D"YES" # Ripmi driver > if_tap_load=3D"YES" # Load bridge driver. > if_vlan_load=3D"YES" # Load vlan driver. > # Turn on Xen (BB 10/08/2019) > xen_kernel=3D"/boot/xen" > xen_cmdline=3D"dom0_mem=3D4096m dom0_max_vcpus=3D4 dom0=3Dpvh,verbose > console=3Dcom2,vga iommu=3Dverbose,debug sync_console=3Dtrue com2=3D11520= 0,8n1 > guest_loglvl=3Dall loglvl=3Dall" > > > > (XEN) Xen version 4.12.1 (buhrow@) (FreeBSD clang version 6.0.1 > (tags/RELEASE_601/final 335540) (based on LLVM 6.0.1)) debug=3Dn Thu Oct= 3 > 11:34:20 PDT 2019 > (XEN) Latest ChangeSet: > (XEN) Console output is synchronous. > (XEN) Bootloader: FreeBSD Loader > (XEN) Command line: dom0_mem=3D4096m dom0_max_vcpus=3D4 dom0=3Dpvh,verbos= e > console=3Dcom2,vga iommu=3Dverbose,debug sync_console=3Dtrue com2=3D11520= 0,8n1 > guest_loglvl=3Dall loglvl=3Dall > (XEN) Xen image load base address: 0 > (XEN) Video information: > (XEN) VGA is text mode 80x25, font 8x16 > (XEN) VBE/DDC methods: none; EDID transfer time: 0 seconds > (XEN) EDID info not retrieved because no DDC retrieval method detected > (XEN) Disc information: > (XEN) Found 4 MBR signatures > (XEN) Found 4 EDD information structures > (XEN) Xen-e820 RAM map: > (XEN) 0000000000000000 - 0000000000098400 (usable) > (XEN) 0000000000098400 - 00000000000a0000 (reserved) > (XEN) 00000000000e0000 - 0000000000100000 (reserved) > (XEN) 0000000000100000 - 00000000bf780000 (usable) > (XEN) 00000000bf78e000 - 00000000bf790000 type 9 > (XEN) 00000000bf790000 - 00000000bf79e000 (ACPI data) > (XEN) 00000000bf79e000 - 00000000bf7d0000 (ACPI NVS) > (XEN) 00000000bf7d0000 - 00000000bf7e0000 (reserved) > (XEN) 00000000bf7ec000 - 00000000c0000000 (reserved) > (XEN) 00000000e0000000 - 00000000f0000000 (reserved) > (XEN) 00000000fee00000 - 00000000fee01000 (reserved) > (XEN) 00000000ffc00000 - 0000000100000000 (reserved) > (XEN) 0000000100000000 - 0000001840000000 (usable) > (XEN) New Xen image base address: 0xbf000000 > (XEN) ACPI: RSDP 000FACE0, 0024 (r2 ACPIAM) > (XEN) ACPI: XSDT BF790100, 008C (r1 SMCI 20120803 MSFT 9= 7) > (XEN) ACPI: FACP BF790290, 00F4 (r3 080312 FACP1521 20120803 MSFT 9= 7) > (XEN) ACPI: DSDT BF7906A0, 6580 (r1 10600 10600000 0 INTL 2005111= 7) > (XEN) ACPI: FACS BF79E000, 0040 > (XEN) ACPI: APIC BF790390, 011E (r1 080312 APIC1521 20120803 MSFT 9= 7) > (XEN) ACPI: MCFG BF7904B0, 003C (r1 080312 OEMMCFG 20120803 MSFT 9= 7) > (XEN) ACPI: SLIT BF7904F0, 0030 (r1 080312 OEMSLIT 20120803 MSFT 9= 7) > (XEN) ACPI: OEMB BF79E040, 0086 (r1 080312 OEMB1521 20120803 MSFT 9= 7) > (XEN) ACPI: SRAT BF79A6A0, 01D0 (r1 080312 OEMSRAT 1 INTL = 1) > (XEN) ACPI: HPET BF79A870, 0038 (r1 080312 OEMHPET 20120803 MSFT 9= 7) > (XEN) ACPI: DMAR BF79E0D0, 0130 (r1 AMI OEMDMAR 1 MSFT 9= 7) > (XEN) ACPI: SSDT BF7A1710, 0363 (r1 DpgPmm CpuPm 12 INTL 2005111= 7) > (XEN) ACPI: EINJ BF79A8B0, 0130 (r1 AMIER AMI_EINJ 20120803 MSFT 9= 7) > (XEN) ACPI: BERT BF79AA40, 0030 (r1 AMIER AMI_BERT 20120803 MSFT 9= 7) > (XEN) ACPI: ERST BF79AA70, 01B0 (r1 AMIER AMI_ERST 20120803 MSFT 9= 7) > (XEN) ACPI: HEST BF79AC20, 00A8 (r1 AMIER ABC_HEST 20120803 MSFT 9= 7) > (XEN) System RAM: 98295MB (100654176kB) > (XEN) SRAT: PXM 0 -> APIC 00 -> Node 0 > (XEN) SRAT: PXM 0 -> APIC 02 -> Node 0 > (XEN) SRAT: PXM 0 -> APIC 04 -> Node 0 > (XEN) SRAT: PXM 0 -> APIC 06 -> Node 0 > (XEN) SRAT: PXM 0 -> APIC 01 -> Node 0 > (XEN) SRAT: PXM 0 -> APIC 03 -> Node 0 > (XEN) SRAT: PXM 0 -> APIC 05 -> Node 0 > (XEN) SRAT: PXM 0 -> APIC 07 -> Node 0 > (XEN) SRAT: PXM 1 -> APIC 10 -> Node 1 > (XEN) SRAT: PXM 1 -> APIC 12 -> Node 1 > (XEN) SRAT: PXM 1 -> APIC 14 -> Node 1 > (XEN) SRAT: PXM 1 -> APIC 16 -> Node 1 > (XEN) SRAT: PXM 1 -> APIC 11 -> Node 1 > (XEN) SRAT: PXM 1 -> APIC 13 -> Node 1 > (XEN) SRAT: PXM 1 -> APIC 15 -> Node 1 > (XEN) SRAT: PXM 1 -> APIC 17 -> Node 1 > (XEN) SRAT: Node 0 PXM 0 0-a0000 > (XEN) SRAT: Node 0 PXM 0 100000-c0000000 > (XEN) SRAT: Node 0 PXM 0 100000000-c40000000 > (XEN) SRAT: Node 1 PXM 1 c40000000-1840000000 > (XEN) NUMA: Allocated memnodemap from 183e1b7000 - 183e1d0000 > (XEN) NUMA: Using 8 for the hash shift. > (XEN) Domain heap initialised DMA width 32 bits > (XEN) CPU Vendor: Intel, Family 6 (0x6), Model 26 (0x1a), Stepping 5 (raw > 000106a5) > (XEN) found SMP MP-table at 000ff780 > (XEN) DMI present. > (XEN) Using APIC driver default > (XEN) ACPI: PM-Timer IO Port: 0x808 (32 bits) > (XEN) ACPI: SLEEP INFO: pm1x_cnt[1:804,1:0], pm1x_evt[1:800,1:0] > (XEN) ACPI: wakeup_vec[bf79e00c], vec_size[20] > (XEN) ACPI: Local APIC address 0xfee00000 > (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled) > (XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x10] enabled) > (XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x02] enabled) > (XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x12] enabled) > (XEN) ACPI: LAPIC (acpi_id[0x05] lapic_id[0x04] enabled) > (XEN) ACPI: LAPIC (acpi_id[0x06] lapic_id[0x14] enabled) > (XEN) ACPI: LAPIC (acpi_id[0x07] lapic_id[0x06] enabled) > (XEN) ACPI: LAPIC (acpi_id[0x08] lapic_id[0x16] enabled) > (XEN) ACPI: LAPIC (acpi_id[0x09] lapic_id[0x01] enabled) > (XEN) ACPI: LAPIC (acpi_id[0x0a] lapic_id[0x11] enabled) > (XEN) ACPI: LAPIC (acpi_id[0x0b] lapic_id[0x03] enabled) > (XEN) ACPI: LAPIC (acpi_id[0x0c] lapic_id[0x13] enabled) > (XEN) ACPI: LAPIC (acpi_id[0x0d] lapic_id[0x05] enabled) > (XEN) ACPI: LAPIC (acpi_id[0x0e] lapic_id[0x15] enabled) > (XEN) ACPI: LAPIC (acpi_id[0x0f] lapic_id[0x07] enabled) > (XEN) ACPI: LAPIC (acpi_id[0x10] lapic_id[0x17] enabled) > (XEN) ACPI: LAPIC (acpi_id[0x11] lapic_id[0x90] disabled) > (XEN) ACPI: LAPIC (acpi_id[0x12] lapic_id[0x91] disabled) > (XEN) ACPI: LAPIC (acpi_id[0x13] lapic_id[0x92] disabled) > (XEN) ACPI: LAPIC (acpi_id[0x14] lapic_id[0x93] disabled) > (XEN) ACPI: LAPIC (acpi_id[0x15] lapic_id[0x94] disabled) > (XEN) ACPI: LAPIC (acpi_id[0x16] lapic_id[0x95] disabled) > (XEN) ACPI: LAPIC (acpi_id[0x17] lapic_id[0x96] disabled) > (XEN) ACPI: LAPIC (acpi_id[0x18] lapic_id[0x97] disabled) > (XEN) ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) > (XEN) Overriding APIC driver with bigsmp > (XEN) ACPI: IOAPIC (id[0x08] address[0xfec00000] gsi_base[0]) > (XEN) IOAPIC[0]: apic_id 8, version 32, address 0xfec00000, GSI 0-23 > (XEN) ACPI: IOAPIC (id[0x09] address[0xfec8a000] gsi_base[24]) > (XEN) IOAPIC[1]: apic_id 9, version 32, address 0xfec8a000, GSI 24-47 > (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) > (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) > (XEN) ACPI: IRQ0 used by override. > (XEN) ACPI: IRQ2 used by override. > (XEN) ACPI: IRQ9 used by override. > (XEN) Enabling APIC mode: Phys. Using 2 I/O APICs > (XEN) ACPI: HPET id: 0x8086a301 base: 0xfed00000 > (XEN) PCI: MCFG configuration 0: base e0000000 segment 0000 buses 00 - ff > (XEN) PCI: MCFG area at e0000000 reserved in E820 > (XEN) PCI: Using MCFG for segment 0000 bus 00-ff > (XEN) [VT-D]Host address width 40 > (XEN) [VT-D]found ACPI_DMAR_DRHD: > (XEN) [VT-D] dmaru->address =3D fbffe000 > (XEN) [VT-D]drhd->address =3D fbffe000 iommu->reg =3D ffff82c00021d000 > (XEN) [VT-D]cap =3D c90780106f0462 ecap =3D f020fe > (XEN) [VT-D] IOAPIC: 0000:f0:1f.7 > (XEN) [VT-D] IOAPIC: 0000:00:13.0 > (XEN) [VT-D] flags: INCLUDE_ALL > (XEN) [VT-D]found ACPI_DMAR_RMRR: > (XEN) [VT-D] endpoint: 0000:00:1d.0 > (XEN) [VT-D] endpoint: 0000:00:1d.1 > (XEN) [VT-D] endpoint: 0000:00:1d.2 > (XEN) [VT-D] endpoint: 0000:00:1d.7 > (XEN) [VT-D] endpoint: 0000:00:1a.0 > (XEN) [VT-D] endpoint: 0000:00:1a.1 > (XEN) [VT-D] endpoint: 0000:00:1a.2 > (XEN) [VT-D] endpoint: 0000:00:1a.7 > (XEN) [VT-D]found ACPI_DMAR_RMRR: > (XEN) [VT-D] endpoint: 0000:00:1d.0 > (XEN) [VT-D] endpoint: 0000:00:1d.1 > (XEN) [VT-D] endpoint: 0000:00:1d.2 > (XEN) [VT-D] endpoint: 0000:00:1d.7 > (XEN) [VT-D] endpoint: 0000:00:1a.0 > (XEN) [VT-D] endpoint: 0000:00:1a.1 > (XEN) [VT-D] endpoint: 0000:00:1a.2 > (XEN) [VT-D] endpoint: 0000:00:1a.7 > (XEN) [VT-D]found ACPI_DMAR_ATSR: > (XEN) [VT-D] atsru->all_ports: 0 > (XEN) [VT-D] bridge: 0000:00:01.0 start=3D0 sec=3D1 sub=3D1 > (XEN) [VT-D] bridge: 0000:00:03.0 start=3D0 sec=3D2 sub=3D2 > (XEN) [VT-D] bridge: 0000:00:05.0 start=3D0 sec=3D3 sub=3D3 > (XEN) [VT-D] bridge: 0000:00:07.0 start=3D0 sec=3D4 sub=3D4 > (XEN) [VT-D] bridge: 0000:00:09.0 start=3D0 sec=3D5 sub=3D5 > (XEN) Failed to get Error Log Address Range. > (XEN) HEST: Table parsing has been initialized > (XEN) Using ACPI (MADT) for SMP configuration information > (XEN) SMP: Allowing 24 CPUs (8 hotplug CPUs) > (XEN) IRQ limits: 48 GSI, 3040 MSI/MSI-X > (XEN) CPU0: Intel machine check reporting enabled > (XEN) Speculative mitigation facilities: > (XEN) Hardware features: > (XEN) Compiled-in support: SHADOW_PAGING > (XEN) Xen settings: BTI-Thunk N/A, SPEC_CTRL: No, Other: > (XEN) L1TF: believed vulnerable, maxphysaddr L1D 44, CPUID 40, Safe > address 10000000000 > (XEN) Support for HVM VMs: RSB EAGER_FPU > (XEN) Support for PV VMs: RSB EAGER_FPU > (XEN) XPTI (64-bit PV only): Dom0 enabled, DomU enabled (without PCID) > (XEN) PV L1TF shadowing: Dom0 disabled, DomU enabled > (XEN) Using scheduler: SMP Credit Scheduler rev2 (credit2) > (XEN) Initializing Credit2 scheduler > (XEN) load_precision_shift: 18 > (XEN) load_window_shift: 30 > (XEN) underload_balance_tolerance: 0 > (XEN) overload_balance_tolerance: -3 > (XEN) runqueues arrangement: socket > (XEN) cap enforcement granularity: 10ms > (XEN) load tracking window length 1073741824 ns > (XEN) Adding cpu 0 to runqueue 0 > (XEN) First cpu on runqueue, activating > (XEN) Platform timer is 14.318MHz HPET > (XEN) Detected 2266.768 MHz processor. > (XEN) Initing memory sharing. > (XEN) alt table ffff82d0804309b8 -> ffff82d0804318d0 > (XEN) Intel VT-d iommu 0 supported page sizes: 4kB. > (XEN) Intel VT-d Snoop Control enabled. > (XEN) Intel VT-d Dom0 DMA Passthrough not enabled. > (XEN) Intel VT-d Queued Invalidation enabled. > (XEN) Intel VT-d Interrupt Remapping enabled. > (XEN) Intel VT-d Posted Interrupt not enabled. > (XEN) Intel VT-d Shared EPT tables not enabled. > (XEN) I/O virtualisation enabled > (XEN) - Dom0 mode: Relaxed > (XEN) Interrupt remapping enabled > (XEN) nr_sockets: 3 > (XEN) ENABLING IO-APIC IRQs > (XEN) -> Using new ACK method > Can you try to add ioapic_ack=3Dold to the xen command line and report if that makes a difference? Roger.