From owner-freebsd-virtualization@freebsd.org Thu Jan 12 05:39:42 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 59440CAC653 for ; Thu, 12 Jan 2017 05:39:42 +0000 (UTC) (envelope-from soralx@cydem.org) Received: from smtp.triumf.ca (smtp.triumf.ca [142.90.100.188]) by mx1.freebsd.org (Postfix) with ESMTP id 462A2107A; Thu, 12 Jan 2017 05:39:42 +0000 (UTC) (envelope-from soralx@cydem.org) Received: from mscad14 (mscad14.triumf.ca [142.90.115.36]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by smtp.triumf.ca (Postfix) with ESMTP id C0614F805; Wed, 11 Jan 2017 21:39:41 -0800 (PST) Date: Wed, 11 Jan 2017 21:39:41 -0800 From: To: Cc: Subject: Re: Issues with GTX960 on CentOS7 using bhyve PCI passthru (FreeBSD 11-RC2) Message-ID: <20170111213941.0789c8ce@mscad14> In-Reply-To: <839b4e54-fd7a-20fd-630f-f8aaee0c79ee@freebsd.org> References: <20170110003332.7cf8ba15@mscad14> <0de7e0fe-5680-b1be-bd57-6bf446c2fd38@talk2dom.com> <0c927784-3e3f-7946-fba9-c25001f4156c@talk2dom.com> <20170110180117.7f246b5a@mscad14> <20170111014544.70670784@mscad14> <93196ea2-5439-49ff-54fd-7b7273bdec85@freebsd.org> <20170111184810.5e9fb07f@mscad14> <839b4e54-fd7a-20fd-630f-f8aaee0c79ee@freebsd.org> X-Mailer: Claws Mail 3.14.1 (GTK+ 2.24.29; amd64-portbld-freebsd9.3) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jan 2017 05:39:42 -0000 > That's a different issue - it's unlikely, if not impossible, to > configure bhyve with enough RAM to hit 37 bits worth where that would > become a problem. No need to worry about that. Well, there may be peripheral cards that have less bits... Anyway, I see what you mean: memory manager can always remap the DMA regions to lower virtual addresses. Good to know it's not an issue. > There are lots of BIOS/UEFI implementations out there that have the > same restriction. In general, there should be no need for a guest to > reprogram device BARs. There are all sorts of situations... But anyway, if it is too much work, then we can forget about this until someone needs it again. > After changing the 64-bit BAR base address, did you still need the > pci=nocrs option for Linux ? I'd hope this would be no longer necessary. No. As expected, the option is no longer needed. BTW, is it [generally] safe to decrease the BAR base address further? My workstation has a CPU with just 36 address bits... > The problem is the knowledge set of graphics/GPU knowhow and > equipment access, and bhyve/PCI programming, are disjoint. The time > I've spent on it has been the inverse, where I feel that I've spent a > half-day doing things that anyone who knew about graphics could get > done in a half-hour :) > > For these type of issues, joint work is best to leverage the > knowledge of both sides. From my point-of-view, the work you've done > has been very helpful. Yeah, we could benefit from more information exchange, I agree. I am trying to do my part to share what I learned; I started with no knowledge of bhyve, PCI, or GPU passthrough a week ago -- so for now my efforts appear useful, but soon the progress will stop, and we will need someone with real knowledge to step in. Incidentally, could someone put a note about that hardcoded BAR base on the bhyve PCI passthrough page [0] if it won't be fixed soon, so many others can play with VGA passthrough meanwhile? [0] https://wiki.freebsd.org/bhyve/pci_passthru > later, > Peter. -- [SorAlx] ridin' VN2000 Classic LT