Date: Tue, 28 Aug 2018 06:54:51 -0700 (PDT) From: "Rodney W. Grimes" <freebsd-rwg@pdx.rh.CN85.dnsmgr.net> To: araujo@freebsd.org Cc: Anish <akgupt3@gmail.com>, freebsd-virtualization@freebsd.org Subject: Re: on bhyve statistics Message-ID: <201808281354.w7SDspUw015651@pdx.rh.CN85.dnsmgr.net> In-Reply-To: <CAOfEmZg3jc3fsvCYg_tpQeXN2u=FYFkPex4O%2BpqgCmx6Vvea3A@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
> On Tue, Aug 28, 2018, 9:38 PM Rodney W. Grimes <
> freebsd-rwg@pdx.rh.cn85.dnsmgr.net> wrote:
>
> > > >> Currently, bhyve does not expose any of these statistics. All the
> > stats
> > > > available through bhyvectl --get-stats seem to be coming from the VMM,
> > > > not from the userspace emulation.
> > >
> > > >That is correct, byhvectl is a diagnostics tool for getting
> > > information from the kernel/vmm module.
> > >
> > > bhyvectl provide stats related to processor vmx/svm from vmm.ko and is
> > the
> > > first thing you want to run for performance regression. It will be nice
> > to
> > > include it as part of bhyve perf tool/dashboard that you are intended to
> > > build.
> >
> > From conversations with Peter Grehan he expressed that bhyvectl is
> > purely a diagnostics tool that should not be depended on by any
> > other tools.
> >
> > If you want to do similiar things you should program to the libvmmapi
> > interface, not bhyvectl.
> >
>
> The libvmmapi is more an internal library for usage with bhyvectl and
> bhyveload than for other purposes, of course it won't stop anybody to
> extend it to achieve other goals.
That is not my understanding at all, libvmmapi is the external and
stable/maintained interface to vmm.ko, it is not at all strickly
for use by bhyvectl or bhyveload or bhyve.
And as such this "api" should be very carefully maintained as
it needs to be caried forward for ever.
> > > -Anish
> > >
> > > On Mon, Aug 27, 2018 at 8:20 AM Rodney W. Grimes <
> > > freebsd-rwg@pdx.rh.cn85.dnsmgr.net> wrote:
> > >
> > > > > Hi list,
> > > > >
> > > > > I'm currently looking at getting the libvirt prometheus exporter[1]
> > to
> > > > > work with libvirt+bhyve. In its current state this doesn't work
> > because
> > > > > at least one of the API calls exposed by libvirt isn't implemented by
> > > > > the libvirt bhyve driver - so I started looking at implementing it.
> > > > >
> > > > > The first API call in question is virDomainBlockStats[2], which
> > returns
> > > > > statistics (number of read and written bytes and ops, respectively).
> > > > >
> > > > > Currently, bhyve does not expose any of these statistics. All the
> > stats
> > > > > available through bhyvectl --get-stats seem to be coming from the
> > VMM,
> > > > > not from the userspace emulation.
> > > >
> > > > That is correct, byhvectl is a diagnostics tool for getting
> > > > information from the kernel/vmm module.
> > > >
> > > > > OTOH, I did see that there are *some*
> > > > > stats being collected in bhyverun.c (see struct bhyvestats {...}
> > > > > stats;). I can't see how these are exposed though - a grep of
> > /usr/src
> > > > > turned up no other uses. Which brings me to the following questions:
> > > > >
> > > > > - are the stats in struct bhyvestats {...} stats exposed or used in
> > any
> > > > > non-obvious way?
> > > >
> > > > Not that I am aware of.
> > > >
> > > > > - architecturally, what would be the best ways to get stats out of
> > the
> > > > > user-space emulations? Off of the top of my head, I could think of
> > the
> > > > > following possibilities:
> > > > > - prometheus exporter
> > > > > - having some socket or pipe to request them
> > > > > - DTrace probes
> > > > >
> > > > > I wouldn't mind implementing any of the above, and so would like to
> > know
> > > > > which of these (or other options) would be the most acceptable, and
> > > > > would appreciate some guidance.
> > > >
> > > > I differ to others on what may be the best way to do this.
> > > >
> > > > > CC'ing novel@ for the libvirt side, and grehan@ for the
> > architectural
> > > > > bhyve questions.
> > > >
> > > > You should replace @grehan with @jhb,@tychon as Peter has moved on,
> > > > and John and Tycho are now the bhyve maintainers. I was going to
> > > > add them, and remove Peter, but I see no cc: anyway, so I am sure
> > > > that they are on the virtualization list though.
> > > >
> > > > > Fabian
> > > > >
> > > > > [1] https://github.com/kumina/libvirt_exporter
> > > > > [2]
> > > >
> > https://libvirt.org/html/libvirt-libvirt-domain.html#virDomainBlockStats
> > > > --
> > > > Rod Grimes
> > > > rgrimes@freebsd.org
> >
> > --
> > Rod Grimes
> > rgrimes@freebsd.org
> > _______________________________________________
> > freebsd-virtualization@freebsd.org mailing list
> > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
> > To unsubscribe, send any mail to "
> > freebsd-virtualization-unsubscribe@freebsd.org"
> >
--
Rod Grimes rgrimes@freebsd.org
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201808281354.w7SDspUw015651>