From owner-freebsd-virtualization@freebsd.org Tue Aug 28 15:31:37 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E125E108F8A1 for ; Tue, 28 Aug 2018 15:31:37 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 70DC389EDF for ; Tue, 28 Aug 2018 15:31:37 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w7SFVXNN016213; Tue, 28 Aug 2018 08:31:33 -0700 (PDT) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w7SFVWfV016212; Tue, 28 Aug 2018 08:31:32 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201808281531.w7SFVWfV016212@pdx.rh.CN85.dnsmgr.net> Subject: Re: on bhyve statistics In-Reply-To: <79435a58-9591-b55a-adca-81e037e904ff@physik.tu-berlin.de> To: Fabian Freyer Date: Tue, 28 Aug 2018 08:31:32 -0700 (PDT) CC: Anish , "freebsd-virtualization@freebsd.org" X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Aug 2018 15:31:38 -0000 > On 8/28/18 3:37 PM, Rodney W. Grimes wrote: > >>>> Currently, bhyve does not expose any of these statistics. All the stats > >>> available through bhyvectl --get-stats seem to be coming from the VMM, > >>> not from the userspace emulation. > >> > >>> That is correct, byhvectl is a diagnostics tool for getting > >> information from the kernel/vmm module. > >> > >> bhyvectl provide stats related to processor vmx/svm from vmm.ko and is the > >> first thing you want to run for performance regression. It will be nice to > >> include it as part of bhyve perf tool/dashboard that you are intended to > >> build. > > > > From conversations with Peter Grehan he expressed that bhyvectl is > > purely a diagnostics tool that should not be depended on by any > > other tools. > > > > If you want to do similiar things you should program to the libvmmapi > > interface, not bhyvectl. > > For context, this is *not* what I'm aiming to do. While bhyvectl just > prints statistics exposed by the VMM through libvmmapi, I'm specifically > asking about instrumenting the userland part, bhyve. > > Also, libvmmapi does have downstream consumers that are not > bhyve{,ctl,load}, e.g. grub2-bhyve, bhyve-multiboot[1], maybe even > xhyve[2] - for performance reasons, I'd prefer scraping metrics through > libvmmapi over shelling out to bhyvectl. Yes, there are other consumers of libvmmapi, why I stressed that it is the api to interface with vmm.ko. It might even make since to have the userland bhyve(8) maintain statistics in libvmmapi for exactly these reasons. > [1] https://github.com/fubarnetes/bhyve-multiboot > [2] https://github.com/mist64/xhyve -- Rod Grimes rgrimes@freebsd.org