From owner-freebsd-amd64@FreeBSD.ORG Thu Jan 23 23:10:02 2014 Return-Path: Delivered-To: freebsd-amd64@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 21B09BAF for ; Thu, 23 Jan 2014 23:10:02 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id F07A81733 for ; Thu, 23 Jan 2014 23:10:01 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id s0NNA1Kg047550 for ; Thu, 23 Jan 2014 23:10:01 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id s0NNA1o8047549; Thu, 23 Jan 2014 23:10:01 GMT (envelope-from gnats) Resent-Date: Thu, 23 Jan 2014 23:10:01 GMT Resent-Message-Id: <201401232310.s0NNA1o8047549@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-amd64@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Simon Matter Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 37040AB0 for ; Thu, 23 Jan 2014 23:09:04 +0000 (UTC) Received: from oldred.freebsd.org (oldred.freebsd.org [IPv6:2001:1900:2254:206a::50:4]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 02C571720 for ; Thu, 23 Jan 2014 23:09:04 +0000 (UTC) Received: from oldred.freebsd.org ([127.0.1.6]) by oldred.freebsd.org (8.14.5/8.14.7) with ESMTP id s0NN93FS080403 for ; Thu, 23 Jan 2014 23:09:03 GMT (envelope-from nobody@oldred.freebsd.org) Received: (from nobody@localhost) by oldred.freebsd.org (8.14.5/8.14.5/Submit) id s0NN93AB080397; Thu, 23 Jan 2014 23:09:03 GMT (envelope-from nobody) Message-Id: <201401232309.s0NN93AB080397@oldred.freebsd.org> Date: Thu, 23 Jan 2014 23:09:03 GMT From: Simon Matter To: freebsd-gnats-submit@FreeBSD.org X-Send-Pr-Version: www-3.1 Subject: amd64/186061: FreeBSD 10 crashes as KVM guest on GNU/Linux on AMD family 10h CPUs X-Mailman-Approved-At: Thu, 23 Jan 2014 23:22:06 +0000 X-BeenThere: freebsd-amd64@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Porting FreeBSD to the AMD64 platform List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jan 2014 23:10:02 -0000 >Number: 186061 >Category: amd64 >Synopsis: FreeBSD 10 crashes as KVM guest on GNU/Linux on AMD family 10h CPUs >Confidential: no >Severity: non-critical >Priority: low >Responsible: freebsd-amd64 >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Thu Jan 23 23:10:01 UTC 2014 >Closed-Date: >Last-Modified: >Originator: Simon Matter >Release: 10.0-RELEASE >Organization: Invoca Systems >Environment: FreeBSD install.corp.invoca.ch 10.0-RELEASE FreeBSD 10.0-RELEASE #0 r260789: Thu Jan 16 22:34:59 UTC 2014 root@snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 >Description: I've installed FreeBSD 10.0-RELEASE as KVM guest on a CentOS 6.5 system. It all works well but as soon as some load hits the FreeBSD guest, the virtual instance just reboots. >How-To-Repeat: Install CentOS 6.5 x86_64 on a HP Microserver (or any other AMD family 10h box like a DL585 G7). Create a KVM guest and install FreeBSD 10 on it. Start some applications on the guest and wait a moment, it will soon boot again. Note: 1) There is no panic on the guest, it just gets virtually switched off and on again. 2) Both x86 archs are affected, i386 and amd64 both crash the same way. >Fix: After the guest has rebooted, dmesg on the host shows this nice message: KVM: Guest triggered AMD Erratum 383 Looking at the FreeBSD kernel code it's clear that the kernel usually detects and deals with AMD Erratum 383 nicely. However, in the situation as a KVM guest, this doesn't work anymore. To make migration of guest between different CPU families possible, Qemu-KVM reports a different family to the guest than the real CPU has. The effect is that the kernels auto detection simply doesn't work anymore. The first thing that came to mind is why not make workaround_erratum383 tunable via sysctl? But maybe it's better to improve auto detection? I found three places where auto detection takes place: x86/x86/mca.c in mca_setup() 723 if (cpu_vendor_id == CPU_VENDOR_AMD && 724 CPUID_TO_FAMILY(cpu_id) == 0x10 && amd10h_L1TP) 725 workaround_erratum383 = 1; i386/i386/pmap.c in pmap_init() 759 if (vm_guest == VM_GUEST_VM && cpu_vendor_id == CPU_VENDOR_AMD && 760 CPUID_TO_FAMILY(cpu_id) == 0x10) 761 workaround_erratum383 = 1; amd64/amd64/pmap.c in pmap_init() 1012 if (vm_guest == VM_GUEST_VM && cpu_vendor_id == CPU_VENDOR_AMD && 1013 CPUID_TO_FAMILY(cpu_id) == 0x10) 1014 workaround_erratum383 = 1; Wouldn't it make sense to change these so that workaround_erratum383 == 1 if a) cpu_vendor_id == CPU_VENDOR_AMD && CPUID_TO_FAMILY(cpu_id) == 0x10) or b) vm_guest == VM_GUEST_VM && cpu_vendor_id == CPU_VENDOR_AMD is true? I don't understand mca_setup() nor pmap_init() but it seems workaround_erratum383 should always be enabled if a) CPU vendor is AMD and CPU family is 10h or b) we are a guest VM and CPU vendor is AMD (no matter which family) Thanks for looking into it! Regards, Simon >Release-Note: >Audit-Trail: >Unformatted: