From owner-svn-src-projects@FreeBSD.ORG Wed Jun 11 17:48:08 2014 Return-Path: Delivered-To: svn-src-projects@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 200E98CD; Wed, 11 Jun 2014 17:48:08 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0DAAC218C; Wed, 11 Jun 2014 17:48:08 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.8/8.14.8) with ESMTP id s5BHm702004176; Wed, 11 Jun 2014 17:48:07 GMT (envelope-from neel@svn.freebsd.org) Received: (from neel@localhost) by svn.freebsd.org (8.14.8/8.14.8/Submit) id s5BHm7Ob004175; Wed, 11 Jun 2014 17:48:07 GMT (envelope-from neel@svn.freebsd.org) Message-Id: <201406111748.s5BHm7Ob004175@svn.freebsd.org> From: Neel Natu Date: Wed, 11 Jun 2014 17:48:07 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r267367 - projects/bhyve_svm/sys/amd64/vmm/amd X-SVN-Group: projects MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jun 2014 17:48:08 -0000 Author: neel Date: Wed Jun 11 17:48:07 2014 New Revision: 267367 URL: http://svnweb.freebsd.org/changeset/base/267367 Log: Disable global interrupts early so all the software state maintained by bhyve is sampled "atomically". Any interrupts after this point will be held pending by the CPU until the guest starts executing and will immediately trigger a #VMEXIT. Reviewed by: Anish Gupta (akgupt3@gmail.com) Modified: projects/bhyve_svm/sys/amd64/vmm/amd/svm.c Modified: projects/bhyve_svm/sys/amd64/vmm/amd/svm.c ============================================================================== --- projects/bhyve_svm/sys/amd64/vmm/amd/svm.c Wed Jun 11 17:19:57 2014 (r267366) +++ projects/bhyve_svm/sys/amd64/vmm/amd/svm.c Wed Jun 11 17:48:07 2014 (r267367) @@ -1156,22 +1156,24 @@ svm_vmrun(void *arg, int vcpu, register_ do { vmexit->inst_length = 0; - /* We are asked to give the cpu by scheduler. */ - if (curthread->td_flags & (TDF_ASTPENDING | TDF_NEEDRESCHED)) { - vmexit->exitcode = VM_EXITCODE_BOGUS; - vmm_stat_incr(vm, vcpu, VMEXIT_ASTPENDING, 1); - VCPU_CTR1(vm, vcpu, - "SVM: ASTPENDING, RIP:0x%lx\n", state->rip); - vmexit->rip = state->rip; - break; - } + + /* + * Disable global interrupts to guarantee atomicity during + * loading of guest state. This includes not only the state + * loaded by the "vmrun" instruction but also software state + * maintained by the hypervisor: suspended and rendezvous + * state, NPT generation number, vlapic interrupts etc. + */ + disable_gintr(); if (vcpu_suspended(suspended_cookie)) { + enable_gintr(); vm_exit_suspended(vm, vcpu, state->rip); break; } if (vcpu_rendezvous_pending(rend_cookie)) { + enable_gintr(); vmexit->exitcode = VM_EXITCODE_RENDEZVOUS; vmm_stat_incr(vm, vcpu, VMEXIT_RENDEZVOUS, 1); VCPU_CTR1(vm, vcpu, @@ -1181,32 +1183,36 @@ svm_vmrun(void *arg, int vcpu, register_ break; } + /* We are asked to give the cpu by scheduler. */ + if (curthread->td_flags & (TDF_ASTPENDING | TDF_NEEDRESCHED)) { + enable_gintr(); + vmexit->exitcode = VM_EXITCODE_BOGUS; + vmm_stat_incr(vm, vcpu, VMEXIT_ASTPENDING, 1); + VCPU_CTR1(vm, vcpu, + "SVM: ASTPENDING, RIP:0x%lx\n", state->rip); + vmexit->rip = state->rip; + break; + } + (void)svm_set_vmcb(svm_get_vmcb(svm_sc, vcpu), svm_sc->asid); - + svm_handle_exitintinfo(svm_sc, vcpu); - + (void)svm_inj_interrupts(svm_sc, vcpu, vlapic); - + /* Change TSS type to available.*/ setup_tss_type(); - /* - * Disable global interrupt to guarantee atomicity - * during loading of guest state. - * See 15.5.1 "Loading guest state" APM2. - */ - disable_gintr(); - /* Launch Virtual Machine. */ svm_launch(vmcb_pa, gctx, hctx); - + /* * Only GDTR and IDTR of host is saved and restore by SVM, * LDTR and TR need to be restored by VMM. * XXX: kernel doesn't use LDT, only user space. */ ltr(GSEL(GPROC0_SEL, SEL_KPL)); - + /* * Guest FS and GS selector are stashed by vmload and vmsave. * Host FS and GS selector are stashed by svm_launch(). @@ -1220,15 +1226,14 @@ svm_vmrun(void *arg, int vcpu, register_ */ wrmsr(MSR_GSBASE, (uint64_t)&__pcpu[vcpustate->lastcpu]); wrmsr(MSR_KGSBASE, (uint64_t)&__pcpu[vcpustate->lastcpu]); - - /* vcpu exit with glbal interrupt disabled. */ + + /* #VMEXIT disables interrupts so re-enable them here. */ enable_gintr(); - + /* Handle #VMEXIT and if required return to user space. */ loop = svm_vmexit(svm_sc, vcpu, vmexit); vcpustate->loop++; vmm_stat_incr(vm, vcpu, VMEXIT_COUNT, 1); - } while (loop); return (0);