Date: Wed, 11 Jan 2017 19:25:42 -0800 (PST) From: "Rodney W. Grimes" <freebsd-rwg@pdx.rh.CN85.dnsmgr.net> To: soralx@cydem.org Cc: grehan@freebsd.org, freebsd-virtualization@freebsd.org Subject: Re: Issues with GTX960 on CentOS7 using bhyve PCI passthru (FreeBSD 11-RC2) Message-ID: <201701120325.v0C3PgU7047029@pdx.rh.CN85.dnsmgr.net> In-Reply-To: <20170111184810.5e9fb07f@mscad14>
next in thread | previous in thread | raw e-mail | index | archive | help
> > > Hi, > > > > >> The problem appears to be in the area of assigning memory-mapped > > >> I/O ranges by bhyve for the VGA card to a region outside of the > > >> CPU's addressable space; i.e., bhyve does not check CPUID's > > >> 0x80000008 AL value (0x27 for my CPU, which is 39 bits -- while > > >> bhyve assigns 0xd000000000 & above for the large Prefetch Memory > > >> chunks, which requires 40 address bits). > > > > That's correct - it's a bug in bhyve. > > Baking a proper fix will be more complicated by the fact that PCIe > cards themselves may have limitations. For example, most nVidia GPUs > have 40 bits DMA addressing capability, some 39, an a few (still > modern) ones -- just 37 [ref. nVidia "README" in the driver package]. > > > PCI passthru doesn't allow the BAR values to be modified (this could > > be changed, but it's a lot of work for little gain). > > Removing another signature of detecting virtualization and increasing > compatibility would be negligible gain? Just asking... I don't think we are going to try and defeat the NVidia virtualization checks, and I can probably assure you that they would patch them as fast as we bypassed them. > > > But: > > > # ./nvidia-smi > > > No devices were found > > > dmesg: > > > [ 173.498953] NVRM: RmInitAdapter failed! (0x53:0x3:1856) > > > [ 173.499115] NVRM: rm_init_adapter failed for device bearing > > > minor number 0 > > > > Looks like you're getting close :) > > Hmm, I'm not seeing myself getting much closer here. Do you know > something I don't? ;) I really hope bhyve developers can spare a > bit of time on getting GPU passthrough to work... I know nothing > about these things, and where I waste half a day messing around, > the problem could be fixed in half an hour by someone who knows. This is officially an item on my plate. Functional hardware that works doing VGA passthrough under ESXi was brought up about 2 weeks ago so I am past that stage. I have Quadro FX3800 on a Supermicro X9DAi. You have helped some in uncovering the next set of issues, but my plate is very large, and seems to have grown appendages that are holding all sorts of things :-) I am still coming up to speed on the bhyve code, so it wont be a half hour fix by me, but it well get fixed. > > later, > > Peter. > > -- > [SorAlx] ridin' VN2000 Classic LT > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" -- Rod Grimes rgrimes@freebsd.org
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201701120325.v0C3PgU7047029>