Date: Tue, 28 Aug 2018 22:21:12 +0800 From: Marcelo Araujo <araujobsdport@gmail.com> To: "Rodney W. Grimes" <freebsd-rwg@pdx.rh.cn85.dnsmgr.net> Cc: Marcelo Araujo <araujo@freebsd.org>, Anish <akgupt3@gmail.com>, freebsd-virtualization@freebsd.org Subject: Re: on bhyve statistics Message-ID: <CAOfEmZgyHawaA_1EG5CktwDfgM7gFHs4MXf6GiUNmtr-vT8E2g@mail.gmail.com> In-Reply-To: <201808281354.w7SDspUw015651@pdx.rh.CN85.dnsmgr.net> References: <CAOfEmZg3jc3fsvCYg_tpQeXN2u=FYFkPex4O%2BpqgCmx6Vvea3A@mail.gmail.com> <201808281354.w7SDspUw015651@pdx.rh.CN85.dnsmgr.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, Aug 28, 2018, 9:54 PM Rodney W. Grimes <
freebsd-rwg@pdx.rh.cn85.dnsmgr.net> wrote:
> > On Tue, Aug 28, 2018, 9:38 PM Rodney W. Grimes <
> > freebsd-rwg@pdx.rh.cn85.dnsmgr.net> wrote:
> >
> > > > >> Currently, bhyve does not expose any of these statistics. All the
> > > stats
> > > > > available through bhyvectl --get-stats seem to be coming from the
> VMM,
> > > > > not from the userspace emulation.
> > > >
> > > > >That is correct, byhvectl is a diagnostics tool for getting
> > > > information from the kernel/vmm module.
> > > >
> > > > bhyvectl provide stats related to processor vmx/svm from vmm.ko and
> is
> > > the
> > > > first thing you want to run for performance regression. It will be
> nice
> > > to
> > > > include it as part of bhyve perf tool/dashboard that you are
> intended to
> > > > build.
> > >
> > > From conversations with Peter Grehan he expressed that bhyvectl is
> > > purely a diagnostics tool that should not be depended on by any
> > > other tools.
> > >
> > > If you want to do similiar things you should program to the libvmmapi
> > > interface, not bhyvectl.
> > >
> >
> > The libvmmapi is more an internal library for usage with bhyvectl and
> > bhyveload than for other purposes, of course it won't stop anybody to
> > extend it to achieve other goals.
>
> That is not my understanding at all, libvmmapi is the external and
> stable/maintained interface to vmm.ko, it is not at all strickly
> for use by bhyvectl or bhyveload or bhyve.
>
You need something between userland and vmm.
>
> And as such this "api" should be very carefully maintained as
> it needs to be caried forward for ever.
>
I won't call it as an API provider as such you cannot do all operations
from creating to launch a guest!
> > > > -Anish
> > > >
> > > > On Mon, Aug 27, 2018 at 8:20 AM Rodney W. Grimes <
> > > > freebsd-rwg@pdx.rh.cn85.dnsmgr.net> wrote:
> > > >
> > > > > > Hi list,
> > > > > >
> > > > > > I'm currently looking at getting the libvirt prometheus
> exporter[1]
> > > to
> > > > > > work with libvirt+bhyve. In its current state this doesn't work
> > > because
> > > > > > at least one of the API calls exposed by libvirt isn't
> implemented by
> > > > > > the libvirt bhyve driver - so I started looking at implementing
> it.
> > > > > >
> > > > > > The first API call in question is virDomainBlockStats[2], which
> > > returns
> > > > > > statistics (number of read and written bytes and ops,
> respectively).
> > > > > >
> > > > > > Currently, bhyve does not expose any of these statistics. All the
> > > stats
> > > > > > available through bhyvectl --get-stats seem to be coming from the
> > > VMM,
> > > > > > not from the userspace emulation.
> > > > >
> > > > > That is correct, byhvectl is a diagnostics tool for getting
> > > > > information from the kernel/vmm module.
> > > > >
> > > > > > OTOH, I did see that there are *some*
> > > > > > stats being collected in bhyverun.c (see struct bhyvestats {...}
> > > > > > stats;). I can't see how these are exposed though - a grep of
> > > /usr/src
> > > > > > turned up no other uses. Which brings me to the following
> questions:
> > > > > >
> > > > > > - are the stats in struct bhyvestats {...} stats exposed or used
> in
> > > any
> > > > > > non-obvious way?
> > > > >
> > > > > Not that I am aware of.
> > > > >
> > > > > > - architecturally, what would be the best ways to get stats out
> of
> > > the
> > > > > > user-space emulations? Off of the top of my head, I could
> think of
> > > the
> > > > > > following possibilities:
> > > > > > - prometheus exporter
> > > > > > - having some socket or pipe to request them
> > > > > > - DTrace probes
> > > > > >
> > > > > > I wouldn't mind implementing any of the above, and so would like
> to
> > > know
> > > > > > which of these (or other options) would be the most acceptable,
> and
> > > > > > would appreciate some guidance.
> > > > >
> > > > > I differ to others on what may be the best way to do this.
> > > > >
> > > > > > CC'ing novel@ for the libvirt side, and grehan@ for the
> > > architectural
> > > > > > bhyve questions.
> > > > >
> > > > > You should replace @grehan with @jhb,@tychon as Peter has moved on,
> > > > > and John and Tycho are now the bhyve maintainers. I was going to
> > > > > add them, and remove Peter, but I see no cc: anyway, so I am sure
> > > > > that they are on the virtualization list though.
> > > > >
> > > > > > Fabian
> > > > > >
> > > > > > [1] https://github.com/kumina/libvirt_exporter
> > > > > > [2]
> > > > >
> > >
> https://libvirt.org/html/libvirt-libvirt-domain.html#virDomainBlockStats
> > > > > --
> > > > > Rod Grimes
> > > > > rgrimes@freebsd.org
> > >
> > > --
> > > Rod Grimes
> > > rgrimes@freebsd.org
> > > _______________________________________________
> > > freebsd-virtualization@freebsd.org mailing list
> > > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
> > > To unsubscribe, send any mail to "
> > > freebsd-virtualization-unsubscribe@freebsd.org"
> > >
>
> --
> Rod Grimes
> rgrimes@freebsd.org
>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAOfEmZgyHawaA_1EG5CktwDfgM7gFHs4MXf6GiUNmtr-vT8E2g>