Date: Thu, 27 Aug 2015 10:10:36 +0000 From: Marie <marieheleneka@gmail.com> To: Matt Churchyard <matt.churchyard@userve.net>, Marcus Reid <marcus@blazingdot.com>, Vick Khera <vivek@khera.org> Cc: "freebsd-virtualization@freebsd.org" <freebsd-virtualization@freebsd.org> Subject: Re: Options for zfs inside a VM backed by zfs on the host Message-ID: <CALXRTbeXMGCGVfQn3OuOzC3VM3si7RFyEZba=ww36FoKb_224g@mail.gmail.com> In-Reply-To: <1a6745e27d184bb99eca7fdbdc90c8b5@SERVER.ad.usd-group.com> References: <CALd%2BdcfJ%2BT-f5gk_pim39BSF7nhBqHC3ab7dXgW8fH43VvvhvA@mail.gmail.com> <20150827061044.GA10221@blazingdot.com> <20150827062015.GA10272@blazingdot.com> <1a6745e27d184bb99eca7fdbdc90c8b5@SERVER.ad.usd-group.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, Aug 27, 2015 at 11:42 AM Matt Churchyard via freebsd-virtualization <freebsd-virtualization@freebsd.org> wrote: > > On Wed, Aug 26, 2015 at 11:10:44PM -0700, Marcus Reid wrote: > > On Wed, Aug 26, 2015 at 05:25:52PM -0400, Vick Khera wrote: > > > > Opinions? Preferably well-reasoned ones. :) > > > > > > > However, having the ARC eating up lots of memory twice seems pretty > > > bletcherous. You can probably do some tuning to reduce that, but I > > > never liked tuning the ARC much. > > > I just realized that you can turn primarycache off per-dataset. Does it > make more sense to turn primarycache=none on the zvol on the host, or > on > the datasets in the vm? I'm thinking on the host, but it might be worth > experimenting. > > I'd be very wary of disabling ARC on the main host, it can have pretty > serious side effects. It could possibly be useful in the guest though, as > data should be cached already by ARC on the host, you're just going through > an extra step of reading through the virtual disk driver, and into host > ARC, instead of directly from the guest memory. Would need testing to know > what performance was like and if there are any side effects. > > I do agree that it doesn't seem unnecessary to have any redundancy in the > guest if the host pool is redundant. Save for any glaring bugs in the > virtual disk emulation, you wouldn't expect to get errors on the guest pool > if the host pool is already checksumming the data. > > It's also worth testing with guest ARC enabled but just limited to a > fairly small size, so you're not disabling it entirely, but doing at little > double-caching as possible. > > ZFS features seems perfect for virtual hosts, although it's not ideal that > you have to give up a big chunk of host RAM for ARC. You may also find that > you need to limit host ARC, then only use "MAX_RAM - MY_ARC_LIMIT" for > guests. Otherwise you'll have ZFS and VMs fighting for memory and enough of > us have seen what shouldn't, but unfortunately does happen in that > situation. > > Matt > - > > > Marcus > I've tried this in the past, and found the worst performance penalty was with ARC disabled in guest. I tried with ARC enabled on host and guest, only on host, only on guest. There was a significant performance penalty with either ARC disabled. I'd still recommend to experiment with it on your own to see if the hit is acceptable or not. Shameless plug: I'm working on a project (tunnelfs.io) which should be useful for this use case. :) Unfortunately, there is no ETA on usable code yet. -- Marie Helene Kvello-Aune
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CALXRTbeXMGCGVfQn3OuOzC3VM3si7RFyEZba=ww36FoKb_224g>