From owner-freebsd-virtualization@freebsd.org Sat Dec 2 23:46:36 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1F1EAE6CA96 for ; Sat, 2 Dec 2017 23:46:36 +0000 (UTC) (envelope-from kmacybsd@gmail.com) Received: from mail-ot0-f179.google.com (mail-ot0-f179.google.com [74.125.82.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E4463191D for ; Sat, 2 Dec 2017 23:46:35 +0000 (UTC) (envelope-from kmacybsd@gmail.com) Received: by mail-ot0-f179.google.com with SMTP id j2so11945528ota.13 for ; Sat, 02 Dec 2017 15:46:35 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=/dyrasKuMg848gFOLbiIxOIfeNEEAjW7+kgdcWN6K0g=; b=sgzW58tHoR9zWjhgPtBhm4vHfSo+xf6g8Kz6+gr8RfdeZvcWjMrVfitPeDpV3xqqeN bSEXd+FARTaffSc2N5cuZYtE2MnUz/fw2hygPuMl3a6L5/9UjMMzYtsTwV6T+Rth4+8B HJLGuuOK+CUMdzrgQPAo5rQ2nrDdy1Qu3wL+MaL0cgixIyS6nVXyhzILcOmNqHQ6Q3DS ItQaeQx0y7iiYIeIg7z+q3n+MzQ/aJZP3SvpeozuwRq8kSEnsvvuX5su8JVWSdGTbq0U 36gVscjpYWHNvdqSNU9uVlAyFffzMiQuojLn/rKRYsacYPh2jM9Nn64Fx+IdhuxtVhXH lN/g== X-Gm-Message-State: AJaThX7JlGQ1tJmn7iTUoxx+SzLkyOWiPCp5+yfZS0jMMTzxzlBkWEDo SCZEjC13QbPMOzFhmWfu+BM78aVfb6oNsAkvLmg= X-Google-Smtp-Source: AGs4zMYovQxbSAMyHB4zzZxnezgNHnNYf5o4S7CSlWoiUFwFHhhAPuUnk6ixfBG9SyGWEBGTyXCaiKvwbyIbXWGKsWo= X-Received: by 10.157.45.161 with SMTP id g30mr11263443otb.122.1512257901448; Sat, 02 Dec 2017 15:38:21 -0800 (PST) MIME-Version: 1.0 References: <201712022003.vB2K3cok036623@pdx.rh.CN85.dnsmgr.net> In-Reply-To: <201712022003.vB2K3cok036623@pdx.rh.CN85.dnsmgr.net> From: "K. Macy" Date: Sat, 02 Dec 2017 23:38:11 +0000 Message-ID: Subject: Re: bhyve uses all available memory during IO-intensive operations To: "Rodney W. Grimes" Cc: Shane Ambler , freebsd-virtualization@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Dec 2017 23:46:36 -0000 There was a standards group but now the interfaces used buy the Linux virtio drivers define the de facto standard. As virtual interfaces go they're fairly decent. So all we need is a backend. The one thing FreeBSD doesn't have that I miss is CPU hot plug when running as a guest - or at least a mechanism to be told to stop running on the APs. That would make live migration much simpler. -M On Sat, Dec 2, 2017 at 12:03 Rodney W. Grimes < freebsd-rwg@pdx.rh.cn85.dnsmgr.net> wrote: > > On Fri, Dec 1, 2017 at 20:02 Rodney W. Grimes < > > freebsd-rwg@pdx.rh.cn85.dnsmgr.net> wrote: > > > > > > On 02/12/2017 08:11, Dustin Wenz wrote: > > > > > > > > > > The commit history shows that chyves defaults to -S if you are > > > > > hosting from FreeBSD 10.3 or later. I'm sure they had a reason for > > > > > doing that, but I don't know what that would be. It seems to an > > > > > inefficient use of main memory if you need to run a lot of VMs. > > > > > > > > It sounds like a reasonable solution to a problem. If host memory is > > > > full it swaps some out, so a bhyve might have free mem but some > could be > > > > swapped out by the host. If the bhyve is out of mem, it's system > swaps > > > > to it's disk, so the host swaps it back in so that the bhyve can then > > > > swap it to its disk... > > > > > > > > Wiring bhyve ram might be a reasonable solution as long as the hosts > > > > physical ram isn't over allocated by bhyve guests. > > > > > > > > The best solution would involve a host and guest talking to each > other > > > > about used mem, but that would break the whole virtual machine > illusion. > > > > At the least it would involve a system telling the hardware what > memory > > > > is used and what is not, which just isn't something any system does. > > > > Maybe that is an idea for the vm guest aware systems of the future. > > > > > > Its actually old technology, its called the memory balloon driver, > > > but bhyve does not have that functionality, yet. > > > > > > > > > > The virtio ballon driver is already there. Implementing a kernel backend > > for it would be trivial. In kernel virtio-net and virtio-p9fs backends > are > > already well underway. > > Don't you also need guest front ends for each OS? That is the hard part > from what I have seen of all the memory over commit stuff. Especially when > you get to the part of "this page of memory in this guest is the exact > same thing as that page of memory in that guest". > > But if we had the backend working, and just the FreeBSD guest frontend > it would be a big win for many of us using bhyve with quantities of > FreeBSD guests. > > Are we compativle enough we can use the KVM guest balloning extensions? > > -- > Rod Grimes > rgrimes@freebsd.org >