From owner-svn-src-head@freebsd.org Tue Jul 12 08:43:11 2016 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 159D8B93993; Tue, 12 Jul 2016 08:43:11 +0000 (UTC) (envelope-from royger@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E37381E96; Tue, 12 Jul 2016 08:43:10 +0000 (UTC) (envelope-from royger@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id u6C8hA2a032979; Tue, 12 Jul 2016 08:43:10 GMT (envelope-from royger@FreeBSD.org) Received: (from royger@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id u6C8h9bc032975; Tue, 12 Jul 2016 08:43:09 GMT (envelope-from royger@FreeBSD.org) Message-Id: <201607120843.u6C8h9bc032975@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: royger set sender to royger@FreeBSD.org using -f From: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= Date: Tue, 12 Jul 2016 08:43:09 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r302635 - in head/sys: amd64/include i386/include x86/x86 x86/xen X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Jul 2016 08:43:11 -0000 Author: royger Date: Tue Jul 12 08:43:09 2016 New Revision: 302635 URL: https://svnweb.freebsd.org/changeset/base/302635 Log: xen: automatically disable MSI-X interrupt migration If the hypervisor version is smaller than 4.6.0. Xen commits 74fd00 and 70a3cb are required on the hypervisor side for this to be fixed, and those are only included in 4.6.0, so stay on the safe side and disable MSI-X interrupt migration on anything older than 4.6.0. It should not cause major performance degradation unless a lot of MSI-X interrupts are allocated. Sponsored by: Citrix Systems R&D MFC after: 3 days Reviewed by: jhb Differential revision: https://reviews.freebsd.org/D7148 Modified: head/sys/amd64/include/intr_machdep.h head/sys/i386/include/intr_machdep.h head/sys/x86/x86/msi.c head/sys/x86/xen/hvm.c Modified: head/sys/amd64/include/intr_machdep.h ============================================================================== --- head/sys/amd64/include/intr_machdep.h Tue Jul 12 08:38:03 2016 (r302634) +++ head/sys/amd64/include/intr_machdep.h Tue Jul 12 08:43:09 2016 (r302635) @@ -149,6 +149,8 @@ extern cpuset_t intr_cpus; extern struct mtx icu_lock; extern int elcr_found; +extern int msix_disable_migration; + #ifndef DEV_ATPIC void atpic_reset(void); #endif Modified: head/sys/i386/include/intr_machdep.h ============================================================================== --- head/sys/i386/include/intr_machdep.h Tue Jul 12 08:38:03 2016 (r302634) +++ head/sys/i386/include/intr_machdep.h Tue Jul 12 08:43:09 2016 (r302635) @@ -140,6 +140,8 @@ extern cpuset_t intr_cpus; extern struct mtx icu_lock; extern int elcr_found; +extern int msix_disable_migration; + #ifndef DEV_ATPIC void atpic_reset(void); #endif Modified: head/sys/x86/x86/msi.c ============================================================================== --- head/sys/x86/x86/msi.c Tue Jul 12 08:38:03 2016 (r302634) +++ head/sys/x86/x86/msi.c Tue Jul 12 08:43:09 2016 (r302635) @@ -149,12 +149,16 @@ struct pic msi_pic = { .pic_reprogram_pin = NULL, }; -/* +/** * Xen hypervisors prior to 4.6.0 do not properly handle updates to * enabled MSI-X table entries. Allow migration of MSI-X interrupts - * to be disabled via a tunable. + * to be disabled via a tunable. Values have the following meaning: + * + * -1: automatic detection by FreeBSD + * 0: enable migration + * 1: disable migration */ -static int msix_disable_migration = 0; +int msix_disable_migration = -1; SYSCTL_INT(_machdep, OID_AUTO, disable_msix_migration, CTLFLAG_RDTUN, &msix_disable_migration, 0, "Disable migration of MSI-X interrupts between CPUs"); @@ -312,6 +316,11 @@ msi_init(void) return; } + if (msix_disable_migration == -1) { + /* The default is to allow migration of MSI-X interrupts. */ + msix_disable_migration = 0; + } + msi_enabled = 1; intr_register_pic(&msi_pic); mtx_init(&msi_lock, "msi", NULL, MTX_DEF); Modified: head/sys/x86/xen/hvm.c ============================================================================== --- head/sys/x86/xen/hvm.c Tue Jul 12 08:38:03 2016 (r302634) +++ head/sys/x86/xen/hvm.c Tue Jul 12 08:43:09 2016 (r302635) @@ -134,9 +134,29 @@ xen_hvm_init_hypercall_stubs(enum xen_hv return (ENXIO); if (init_type == XEN_HVM_INIT_COLD) { + int major, minor; + do_cpuid(base + 1, regs); - printf("XEN: Hypervisor version %d.%d detected.\n", - regs[0] >> 16, regs[0] & 0xffff); + + major = regs[0] >> 16; + minor = regs[0] & 0xffff; + printf("XEN: Hypervisor version %d.%d detected.\n", major, + minor); + + if (((major < 4) || (major == 4 && minor <= 5)) && + msix_disable_migration == -1) { + /* + * Xen hypervisors prior to 4.6.0 do not properly + * handle updates to enabled MSI-X table entries, + * so disable MSI-X interrupt migration in that + * case. + */ + if (bootverbose) + printf( +"Disabling MSI-X interrupt migration due to Xen hypervisor bug.\n" +"Set machdep.msix_disable_migration=0 to forcefully enable it.\n"); + msix_disable_migration = 1; + } } /*