From owner-freebsd-virtualization@FreeBSD.ORG Wed Aug 20 16:00:43 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C64FE724; Wed, 20 Aug 2014 16:00:43 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9AAAD3082; Wed, 20 Aug 2014 16:00:43 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 7EEC5B9CE; Wed, 20 Aug 2014 12:00:42 -0400 (EDT) From: John Baldwin To: freebsd-drivers@freebsd.org Subject: Re: Directly reserve an interrupt IDT entry for Hyper-V Date: Wed, 20 Aug 2014 11:31:29 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20140415; KDE/4.5.5; amd64; ; ) References: <3defccd56bc0407c88bdda139b2462ac@BY1PR0301MB0902.namprd03.prod.outlook.com> <53F4A34A.4010807@citrix.com> In-Reply-To: <53F4A34A.4010807@citrix.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Message-Id: <201408201131.29399.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Wed, 20 Aug 2014 12:00:42 -0400 (EDT) Cc: "freebsd-xen@freebsd.org" , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Aug 2014 16:00:43 -0000 On Wednesday, August 20, 2014 9:31:54 am Roger Pau Monn=E9 wrote: > On 20/08/14 11:19, Wei Hu wrote: > > Hello, > >=20 > > Sending to Xen, drivers and virtualization mailing lists since this mig= ht=20 be of interest to the folks on these aliases. > >=20 > > I am working for Microsoft to improve the performance of FreeBSD runnin= g=20 on Hyper-V. Right now I am adding a feature in the vmbus driver which could= =20 handle the host-guest channel communications on all vCPUs simultaneously. I= n=20 order to achieve this, the hypervisor will send same interrupt concurrently= on=20 all the vCPUs. The traditional way on FreeBSD to set up interrupt handling = for=20 devicse, such as calling bus_alloc_resource() to reserve an IRQ line, and t= hen=20 calling bus_setup_intr() to create a vector, doesn't seem to work in this=20 case. It seems if the interrupt is routed via legacy IRQ, it can only be=20 active on one vCPU at a time. In order to allow the same interrupt to be=20 handled on all vCPUs concurrently, all I need is an IDT entry, not an IRQ=20 line. > >=20 > > I checked current FreeBSD code. It looks to me Xen directly uses the=20 vector number IDT_EVTCHN (0x93) to achieve the same purpose. I am proposing= =20 both Xen and Hyper-V share this same vector. Following is a little bit deta= il=20 of my proposal for the changes in the current kernel. > >=20 > >=20 > > 1. In machdep.c: > >=20 > > #ifdef XENHVM > >=20 > > setidt(IDT_EVTCHN, &IDTVEC(xen_intr_upcall), SDT_SYSIGT, SEL_UP= L,=20 0); > >=20 > > #else > >=20 > > setidt(IDT_EVTCHN, &IDTVEC(hv_vmbus_intr), SDT_SYSIGT, SEL_UPL,= =20 0); > >=20 > > #endif > >=20 > > 2. Apic_vector.S > >=20 > > Add IDTVEC(hv_vmbus_intr) to call Hyper-V vmbus interrupt service routi= ne. > >=20 > > Any thoughts, objections and feedbacks are all welcome. >=20 > Hello, >=20 > I don't think using the same IDT vector is the right approach, I would > just pick a different IDT vector and use that for Hyper-V. Using the > same IDT vector (like your suggestion above) would prevent shipping a > kernel with with both Hyper-V and Xen support (like it's done now in > GENERIC). >=20 > Roger. Hmm, can't you make this a runtime check to only call setidt() if you detec= t=20 you are under the appropriate hypervisor? Also, bhyve currently has a hackish way of requesting a free IDT slot. =20 Perhaps it would be best if I added little API to reserve an IDT slot assum= ing=20 that callers could accept a dynamic IDT vector rather than a static one. =2D-=20 John Baldwin