From owner-freebsd-virtualization@freebsd.org Tue Aug 28 14:21:28 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1E354108DD7A for ; Tue, 28 Aug 2018 14:21:28 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: from mail-lf1-x129.google.com (mail-lf1-x129.google.com [IPv6:2a00:1450:4864:20::129]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 76E3A8611C; Tue, 28 Aug 2018 14:21:27 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: by mail-lf1-x129.google.com with SMTP id e23-v6so1512432lfc.13; Tue, 28 Aug 2018 07:21:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:reply-to:from:date:message-id :subject:to:cc; bh=76qVTlSc/PeAq8pDSX1z9dxRNACKvOxWOeffyXuQ68M=; b=hE20d26qTILDdYaIi0MUkcTv0N4KNf/Z2UAPjdf2ALuo7V5VQixXujeh0qAonflIDV Am9l0iTG3xbp9uGHAXK0JwWTbTrGOTiav+ahPn9Y9J2T+tea3hHb1LWQhu4Cd3nuuwIl gOE2LYOR4TjFCTebusdzr2b4mAM6CSq3EHJ8Sd8D7iE416S/G7t+aS2IJS+6VSVhP7eC V/6FUe63mfZ0MT/N8sTFtYfxsFMURhIHj2F1R03uMe2cjxlSIeV/X/LAOnilLwDgWbG9 lE8sX7w/R3y46pumo3pVQmmHEoyokfduvKBKQuZqFWOL5c6bupRSBmeUHrg1sNBVtzJN gXVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:reply-to :from:date:message-id:subject:to:cc; bh=76qVTlSc/PeAq8pDSX1z9dxRNACKvOxWOeffyXuQ68M=; b=pijn/2huM0wIPFqBOXxOyjO8i7hKoEPGy6knG4o2buv8KtBBIsM6G0OiXNG2zDWgVn 4RJwDT+IPfwEP2hioVobbUbVzRO3S9LVfdkob0+y3wfiKXIAfwlFXm5hX6vSujc794yZ oMF76Gh/m4QSCfd+Yh02z0kAfhUPAMDSqKCeufPbl3V+x0qpc4CKaWdoLDG0aVD5Ovw8 2P9I8welVX5aHErPLX7dcEZBLuLPZpPtpU8p5YA/fxW1Ade0y+oMLp74BMjlZrTdbzIl OlQIWwo4gZ0nKT9TstpgdzI5nZpntgLhWRTBOTHoFsUnkTX7k0zG0rCX878hOvHMBQJc 3NUg== X-Gm-Message-State: APzg51BcwjUANKadYEMYZFyuOPrIFXb//7b2ez/APzclHGH92Pqlk62F jQxwxkTNvShcYhaNWULF6oaRfNXd+PQy49YCGgA= X-Google-Smtp-Source: ANB0VdazgjC0awmlFVD9E72Q/fVTIwAVL+ZErN99BWPnJ5BL70WTDXAh2eyom1Cxf3W83Mt21/PGqZJWRBiMaAqpGbE= X-Received: by 2002:a19:54c2:: with SMTP id b63-v6mr1513513lfl.128.1535466085443; Tue, 28 Aug 2018 07:21:25 -0700 (PDT) MIME-Version: 1.0 References: <201808281354.w7SDspUw015651@pdx.rh.CN85.dnsmgr.net> In-Reply-To: <201808281354.w7SDspUw015651@pdx.rh.CN85.dnsmgr.net> Reply-To: araujo@freebsd.org From: Marcelo Araujo Date: Tue, 28 Aug 2018 22:21:12 +0800 Message-ID: Subject: Re: on bhyve statistics To: "Rodney W. Grimes" Cc: Marcelo Araujo , Anish , freebsd-virtualization@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Aug 2018 14:21:28 -0000 On Tue, Aug 28, 2018, 9:54 PM Rodney W. Grimes < freebsd-rwg@pdx.rh.cn85.dnsmgr.net> wrote: > > On Tue, Aug 28, 2018, 9:38 PM Rodney W. Grimes < > > freebsd-rwg@pdx.rh.cn85.dnsmgr.net> wrote: > > > > > > >> Currently, bhyve does not expose any of these statistics. All the > > > stats > > > > > available through bhyvectl --get-stats seem to be coming from the > VMM, > > > > > not from the userspace emulation. > > > > > > > > >That is correct, byhvectl is a diagnostics tool for getting > > > > information from the kernel/vmm module. > > > > > > > > bhyvectl provide stats related to processor vmx/svm from vmm.ko and > is > > > the > > > > first thing you want to run for performance regression. It will be > nice > > > to > > > > include it as part of bhyve perf tool/dashboard that you are > intended to > > > > build. > > > > > > From conversations with Peter Grehan he expressed that bhyvectl is > > > purely a diagnostics tool that should not be depended on by any > > > other tools. > > > > > > If you want to do similiar things you should program to the libvmmapi > > > interface, not bhyvectl. > > > > > > > The libvmmapi is more an internal library for usage with bhyvectl and > > bhyveload than for other purposes, of course it won't stop anybody to > > extend it to achieve other goals. > > That is not my understanding at all, libvmmapi is the external and > stable/maintained interface to vmm.ko, it is not at all strickly > for use by bhyvectl or bhyveload or bhyve. > You need something between userland and vmm. > > And as such this "api" should be very carefully maintained as > it needs to be caried forward for ever. > I won't call it as an API provider as such you cannot do all operations from creating to launch a guest! > > > > -Anish > > > > > > > > On Mon, Aug 27, 2018 at 8:20 AM Rodney W. Grimes < > > > > freebsd-rwg@pdx.rh.cn85.dnsmgr.net> wrote: > > > > > > > > > > Hi list, > > > > > > > > > > > > I'm currently looking at getting the libvirt prometheus > exporter[1] > > > to > > > > > > work with libvirt+bhyve. In its current state this doesn't work > > > because > > > > > > at least one of the API calls exposed by libvirt isn't > implemented by > > > > > > the libvirt bhyve driver - so I started looking at implementing > it. > > > > > > > > > > > > The first API call in question is virDomainBlockStats[2], which > > > returns > > > > > > statistics (number of read and written bytes and ops, > respectively). > > > > > > > > > > > > Currently, bhyve does not expose any of these statistics. All the > > > stats > > > > > > available through bhyvectl --get-stats seem to be coming from the > > > VMM, > > > > > > not from the userspace emulation. > > > > > > > > > > That is correct, byhvectl is a diagnostics tool for getting > > > > > information from the kernel/vmm module. > > > > > > > > > > > OTOH, I did see that there are *some* > > > > > > stats being collected in bhyverun.c (see struct bhyvestats {...} > > > > > > stats;). I can't see how these are exposed though - a grep of > > > /usr/src > > > > > > turned up no other uses. Which brings me to the following > questions: > > > > > > > > > > > > - are the stats in struct bhyvestats {...} stats exposed or used > in > > > any > > > > > > non-obvious way? > > > > > > > > > > Not that I am aware of. > > > > > > > > > > > - architecturally, what would be the best ways to get stats out > of > > > the > > > > > > user-space emulations? Off of the top of my head, I could > think of > > > the > > > > > > following possibilities: > > > > > > - prometheus exporter > > > > > > - having some socket or pipe to request them > > > > > > - DTrace probes > > > > > > > > > > > > I wouldn't mind implementing any of the above, and so would like > to > > > know > > > > > > which of these (or other options) would be the most acceptable, > and > > > > > > would appreciate some guidance. > > > > > > > > > > I differ to others on what may be the best way to do this. > > > > > > > > > > > CC'ing novel@ for the libvirt side, and grehan@ for the > > > architectural > > > > > > bhyve questions. > > > > > > > > > > You should replace @grehan with @jhb,@tychon as Peter has moved on, > > > > > and John and Tycho are now the bhyve maintainers. I was going to > > > > > add them, and remove Peter, but I see no cc: anyway, so I am sure > > > > > that they are on the virtualization list though. > > > > > > > > > > > Fabian > > > > > > > > > > > > [1] https://github.com/kumina/libvirt_exporter > > > > > > [2] > > > > > > > > > https://libvirt.org/html/libvirt-libvirt-domain.html#virDomainBlockStats > > > > > -- > > > > > Rod Grimes > > > > > rgrimes@freebsd.org > > > > > > -- > > > Rod Grimes > > > rgrimes@freebsd.org > > > _______________________________________________ > > > freebsd-virtualization@freebsd.org mailing list > > > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > > > To unsubscribe, send any mail to " > > > freebsd-virtualization-unsubscribe@freebsd.org" > > > > > -- > Rod Grimes > rgrimes@freebsd.org >