Date: Tue, 18 Mar 2014 13:51:47 -0400 From: Ryan Stone <rysto32@gmail.com> To: Neel Natu <neelnatu@gmail.com> Cc: Konstantin Belousov <kostikbel@gmail.com>, "freebsd-hackers@freebsd.org" <freebsd-hackers@freebsd.org> Subject: Re: [PATCH] Support PCIe Alternative RID Interpretation (ARI) Message-ID: <CAFMmRNwCGVhyn5cU29YpsVq44Q5i51C38GVsz33xGeqNyemx0Q@mail.gmail.com> In-Reply-To: <CAFgRE9F632zLseG7MobxgV5CdvD0KyMn28CBSwYqVtZKuLBwRw@mail.gmail.com> References: <CAFMmRNzL3uBZ-djWgpnKi3XDQdq4c1ODAL_8E-Vpy-dPLa-hog@mail.gmail.com> <20140316141216.GA21331@kib.kiev.ua> <CAFMmRNwormaaPXk6rJ-JJGePS6fDNFsdKAfmmW2jGLNRscf1Pw@mail.gmail.com> <CAFgRE9F632zLseG7MobxgV5CdvD0KyMn28CBSwYqVtZKuLBwRw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, Mar 17, 2014 at 5:20 PM, Neel Natu <neelnatu@gmail.com> wrote: > Hi Ryan, > In any case it seems that the VT-d implementation in bhyve will work > fine with ARI enabled devices. There was an assert that would trip for the function number being greater than 8. I've put together the following patch series: http://people.freebsd.org/~rstone/patches/ari/0001-Add-a-method-to-get-the-PCI-RID-for-a-device.patch This adds a method to get the RID that will be consumed by the VT-d drivers. This patch is non-ARI only. http://people.freebsd.org/~rstone/patches/ari/0002-Re-implement-the-DMAR-I-O-MMU-code-in-terms-of-PCI-R.patch This reworks the busdma DMAR code to work in terms of RIDs where necessary. This should be a no-op. I tested this with hw.dmar.enable=1 on a Nehalem with the em driver and a Sandy Bridge with the igb and ixgbe driver and was able to pass traffic. http://people.freebsd.org/~rstone/patches/ari/0003-Re-write-bhyve-s-I-O-MMU-handling-in-terms-of-PCI-RI.patch Same thing, but for bhyve. I'm not sure that passing the rid down to the CPU-dependent layer is the right thing to do, because I'm not sure what the AMD VT-d equivalent requires. Should I just pass down the entire device_t and let the CPU layer deal with it? I tested loading vmm.ko with a device assigned to the ppt driver but I didn't go as far as starting a VM and using PCI passthrough. (Also, as you'd probably expected doing this with hw.dmar.enable=1 causes all hell to break loose). http://people.freebsd.org/~rstone/patches/ari/0004-Add-support-for-PCIe-ARI.patch This is a slightly reworked version of the previous patch. The main difference are that there is a new implementation of pcib_get_rid that understands ARI RIDs. I also fixed a bug where the default implementation of pcib_numslots was not actually being used because I misspelled DEFAULT as default in pcib_if.m. http://people.freebsd.org/~rstone/patches/ari/0005-Print-status-of-ARI-capability-in-pciconf-c.patch This makes pciconf -c dump the status of ARI on PCIe downstream ports.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAFMmRNwCGVhyn5cU29YpsVq44Q5i51C38GVsz33xGeqNyemx0Q>