From owner-freebsd-virtualization@FreeBSD.ORG Tue Jun 3 01:55:54 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3A9AFD9E for ; Tue, 3 Jun 2014 01:55:54 +0000 (UTC) Received: from mail-qa0-x22b.google.com (mail-qa0-x22b.google.com [IPv6:2607:f8b0:400d:c00::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F1AD82C26 for ; Tue, 3 Jun 2014 01:55:53 +0000 (UTC) Received: by mail-qa0-f43.google.com with SMTP id m5so3997096qaj.2 for ; Mon, 02 Jun 2014 18:55:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=61Cel0scjIfD/QVE1v2n7EBIJx6MuO9jaWzjzxvBONA=; b=C4gDbu9Ds1FCXIsIOwMdMNc+aNs6xzB+4YDfte3vdIFfuVyKhUI+ealkS6VprdY7j+ 4AK2h362YEn8qn4wblKBCZtGObglLH/5d5uTvbVMklDyO8ZfmqxNj0+cLUDag+73CfD0 r0HZlQ20+k7AcrPRRA6B0mbjjZuY1nAZ7oxp4HVaa1TiJ2o8kt2G8yDomFIOYq0wdKr6 Ue0Pvwndr8mo8TuetImeX1JCmSkQalSAqEgKP7+WYqu8bpC8Ur7I8g6sza1lxKnB/Jsa p4DplszOJfzqBbZRtATxjhuWEfPQ0PITNepwSGOeRWwOwgbLylTCPK26uPZfpAsB4oPv b5iw== MIME-Version: 1.0 X-Received: by 10.140.96.162 with SMTP id k31mr50654803qge.38.1401760553141; Mon, 02 Jun 2014 18:55:53 -0700 (PDT) Received: by 10.140.48.37 with HTTP; Mon, 2 Jun 2014 18:55:53 -0700 (PDT) In-Reply-To: References: Date: Mon, 2 Jun 2014 18:55:53 -0700 Message-ID: Subject: Re: SR-IOV Patch Series 2/7: bhyve integration From: Neel Natu To: Ryan Stone Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Jun 2014 01:55:54 -0000 Hi Ryan, On Mon, May 26, 2014 at 6:51 PM, Ryan Stone wrote: > The bhyve work to interoperate is quite simple. PCI Passthrough > through the ppt driver works just fine with SR-IOV VFs. The three > changes are: > > - allow ppt devices to detach. This would happen if the administrator > destroyed VFs with iovctl -D > - allow the SR-IOV infrastructure to force the ppt driver to attach to > certain devices by using the special BUS_PROBE_NOWILDCARD return value > from device_probe(). This will have no effect on regular devices, but > the SR-IOV infrastructure can create VF devices in a way where > BUS_PROBE_NOWILDCARD will match the device. This allows passthrough > of a given VF to be configured through iovctl (see ioctl.conf(5) for > an example of how to do this). > - Currently if vmm.ko is loaded and no ppt devices are present, the > IOMMU is not enabled. If VFs are subsequently created as passthrough > devices it will not be possible to pass them through to VMs. This > patch adds a new tunable that will force the IOMMU to be enabled. > > Note: Currently ppt suffers from a lack of locking. This is now a > problem because ppt devices can be destroyed, so there would be races > here. I tried to address this by adding a mutex but that failed in > ways that I could not figure out how to resolve (basically, some parts > of ppt are called into from within a critical section, so a spinlock > is needed, but other parts of ppt call into the VM subsystem to do > M_WAITOK allocations, so an sx lock is needed. It wasn't at all > obvious to me how to resolve that conflict, so I'm hoping that a more > experienced bhyve person has an idea). > Your changes look good. Also, yes, locking is now an issue. Let me try a couple of things and see how to fix this properly. best Neel > > http://people.freebsd.org/~rstone/patches/iov/0004-Allow-passthrough-devices-to-be-hinted.patch > [PATCH 04/21] Allow passthrough devices to be hinted. > > Allow the ppt driver to attach to devices that were hinted to be > passthrough devices by the PCI code creating them with a driver > name of "ppt". > > Add a tunable that allows the IOMMU to be forced to be used. With > SR-IOV passthrough devices the VFs may be created after vmm.ko is > loaded. The current code will not initialize the IOMMU in that > case, meaning that the passthrough devices can't actually be used. > --- > sys/amd64/vmm/io/ppt.c | 72 +++++++++++++++++++++++++++----------------------- > sys/amd64/vmm/vmm.c | 7 ++++- > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org"