Date: Thu, 20 Mar 2014 09:47:03 -0700 From: Neel Natu <neelnatu@gmail.com> To: Ryan Stone <rysto32@gmail.com> Cc: Konstantin Belousov <kostikbel@gmail.com>, "freebsd-hackers@freebsd.org" <freebsd-hackers@freebsd.org> Subject: Re: [PATCH] Support PCIe Alternative RID Interpretation (ARI) Message-ID: <CAFgRE9H34NaETwTTe1ryA-gi=ucj8GA8hrguab84YhNi70hqYA@mail.gmail.com> In-Reply-To: <CAFMmRNwCGVhyn5cU29YpsVq44Q5i51C38GVsz33xGeqNyemx0Q@mail.gmail.com> References: <CAFMmRNzL3uBZ-djWgpnKi3XDQdq4c1ODAL_8E-Vpy-dPLa-hog@mail.gmail.com> <20140316141216.GA21331@kib.kiev.ua> <CAFMmRNwormaaPXk6rJ-JJGePS6fDNFsdKAfmmW2jGLNRscf1Pw@mail.gmail.com> <CAFgRE9F632zLseG7MobxgV5CdvD0KyMn28CBSwYqVtZKuLBwRw@mail.gmail.com> <CAFMmRNwCGVhyn5cU29YpsVq44Q5i51C38GVsz33xGeqNyemx0Q@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi Ryan, On Tue, Mar 18, 2014 at 10:51 AM, Ryan Stone <rysto32@gmail.com> wrote: > On Mon, Mar 17, 2014 at 5:20 PM, Neel Natu <neelnatu@gmail.com> wrote: >> Hi Ryan, >> In any case it seems that the VT-d implementation in bhyve will work >> fine with ARI enabled devices. > > There was an assert that would trip for the function number being > greater than 8. > > > I've put together the following patch series: > > http://people.freebsd.org/~rstone/patches/ari/0001-Add-a-method-to-get-the-PCI-RID-for-a-device.patch > > This adds a method to get the RID that will be consumed by the VT-d > drivers. This patch is non-ARI only. > > > http://people.freebsd.org/~rstone/patches/ari/0002-Re-implement-the-DMAR-I-O-MMU-code-in-terms-of-PCI-R.patch > > This reworks the busdma DMAR code to work in terms of RIDs where > necessary. This should be a no-op. I tested this with > hw.dmar.enable=1 on a Nehalem with the em driver and a Sandy Bridge > with the igb and ixgbe driver and was able to pass traffic. > > > http://people.freebsd.org/~rstone/patches/ari/0003-Re-write-bhyve-s-I-O-MMU-handling-in-terms-of-PCI-RI.patch > > Same thing, but for bhyve. I'm not sure that passing the rid down to > the CPU-dependent layer is the right thing to do, because I'm not sure > what the AMD VT-d equivalent requires. Should I just pass down the > entire device_t and let the CPU layer deal with it? I tested loading > vmm.ko with a device assigned to the ppt driver but I didn't go as far > as starting a VM and using PCI passthrough. > The bhyve portion of the patch looks fine to me. A cursory read of the AMD IOMMU spec suggests that the translation table is indexed using the 16-bit RID so passing it into the CPU-dependent layer will work fine. best Neel > (Also, as you'd probably expected doing this with hw.dmar.enable=1 > causes all hell to break loose). > > > http://people.freebsd.org/~rstone/patches/ari/0004-Add-support-for-PCIe-ARI.patch > > This is a slightly reworked version of the previous patch. The main > difference are that there is a new implementation of pcib_get_rid that > understands ARI RIDs. I also fixed a bug where the default > implementation of pcib_numslots was not actually being used because I > misspelled DEFAULT as default in pcib_if.m. > > > http://people.freebsd.org/~rstone/patches/ari/0005-Print-status-of-ARI-capability-in-pciconf-c.patch > > This makes pciconf -c dump the status of ARI on PCIe downstream ports.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAFgRE9H34NaETwTTe1ryA-gi=ucj8GA8hrguab84YhNi70hqYA>