Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 30 Nov 2017 21:10:26 -0600
From:      Dustin Wenz <dustinwenz@ebureau.com>
To:        Allan Jude <allanjude@freebsd.org>
Cc:        freebsd-virtualization@freebsd.org
Subject:   Re: bhyve uses all available memory during IO-intensive operations
Message-ID:  <F633EDFD-E9DA-47A3-AEFD-ED9C499FBD29@ebureau.com>
In-Reply-To: <a324b830-b941-2767-2272-957da680d997@freebsd.org>
References:  <F4E35CB9-30F9-4C63-B4CC-F8ADC9947E3C@ebureau.com> <a324b830-b941-2767-2272-957da680d997@freebsd.org>

next in thread | previous in thread | raw e-mail | index | archive | help
I am using a zvol as the storage for the VM, and I do not have any ARC limit=
s set. However, the bhyve process itself ends up grabbing the vast majority o=
f memory.=20

I=E2=80=99ll run a test tomorrow to get the exact output from top.

   - .Dustin

> On Nov 30, 2017, at 5:28 PM, Allan Jude <allanjude@freebsd.org> wrote:
>=20
>> On 11/30/2017 18:15, Dustin Wenz wrote:
>> I'm using chyves on FreeBSD 11.1 RELEASE to manage a few VMs (guest OS is=
 also FreeBSD 11.1). Their sole purpose is to house some medium-sized Postgr=
es databases (100-200GB). The host system has 64GB of real memory and 112GB o=
f swap. I have configured each guest to only use 16GB of memory, yet while d=
oing my initial database imports in the VMs, bhyve will quickly grow to use a=
ll available system memory and then be killed by the kernel:
>>=20
>>    kernel: swap_pager: I/O error - pageout failed; blkno 1735,size 4096, e=
rror 12
>>    kernel: swap_pager: I/O error - pageout failed; blkno 1610,size 4096, e=
rror 12
>>    kernel: swap_pager: I/O error - pageout failed; blkno 1763,size 4096, e=
rror 12
>>    kernel: pid 41123 (bhyve), uid 0, was killed: out of swap space
>>=20
>> The OOM condition seems related to doing moderate IO within the VM, thoug=
h nothing within the VM itself shows high memory usage. This is the chyves c=
onfig for one of them:
>>=20
>>    bargs                      -A -H -P -S
>>    bhyve_disk_type            virtio-blk
>>    bhyve_net_type             virtio-net
>>    bhyveload_flags
>>    chyves_guest_version       0300
>>    cpu                        4
>>    creation                   Created on Mon Oct 23 16:17:04 CDT 2017 by c=
hyves v0.2.0 2016/09/11 using __create()
>>    loader                     bhyveload
>>    net_ifaces                 tap51
>>    os                         default
>>    ram                        16G
>>    rcboot                     0
>>    revert_to_snapshot
>>    revert_to_snapshot_method  off
>>    serial                     nmdm51
>>    template                   no
>>    uuid                       8495a130-b837-11e7-b092-0025909a8b56
>>=20
>>=20
>> I've also tried using different bhyve_disk_types, with no improvement. Ho=
w is it that bhyve can use far more memory that I'm specifying?
>>=20
>>    - .Dustin
>>=20
>=20
> Can you show 'top' output. What makes you think bhyve is using the
> memory? Are you using ZFS? Have you limited the vfs.zfs.arc_max to leave
> some free RAM for the bhyve instances?
>=20
> --=20
> Allan Jude
> _______________________________________________
> freebsd-virtualization@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
> To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freeb=
sd.org"



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?F633EDFD-E9DA-47A3-AEFD-ED9C499FBD29>