Skip site navigation (1)Skip section navigation (2)
Date:      Fri, 16 Oct 2020 18:54:06 +0100
From:      Jon Schneider <freebsd@jschneider.net>
To:        doug@safeport.com
Cc:        freebsd-questions@freebsd.org, Twingly Customer Support <team@twingly.com>
Subject:   Re: FreeBSD using swap even though there's a lot of free memory
Message-ID:  <05399132-C7A7-4B96-B6E4-D469CCDC70D5@jschneider.net>

Next in thread | Raw E-Mail | Index | Archive | Help
=EF=BB=BFSunOS 4 used to (speculatively) page stuff out. Probably just indic=
ates relatively unused pages.

Jon

> On 16 Oct 2020, at 18:50, doug <doug@fledge.watson.org> wrote:
>=20
> =EF=BB=BF
>=20
>> On Thu, 15 Oct 2020, Jon Schneider wrote:
>> top -w -oswap
>> seems to report the right thing. Would be interesting if it's _not_ MySQL=
.
>> Jon
>>> On 15/10/2020 15:23, Twingly Customer Support wrote:
>>> Hi,
>>> We have a server running FreeBSD 12.1-RELEASE-p10. We currently have a p=
roblem where FreeBSD starting to swap when running ZFS scrub, even though we=
 have ~70G of free memory.
>>> This did not happen before when running FreeBSD 11.3 for example. It sta=
rted happening at approximately the time we upgraded from 12.1-RELEASE-p5 to=
 12.1-RELEASE-p6, but if the upgrade is the cause of the problem is unclear,=
 though FreeBSD never swapped for us before that. "Laundry" memory was not s=
omething we saw before either, it started to appear at the same time as Free=
BSD started swapping.
>>> Eventually, after scrubbing a few times, the swap becomes full and we st=
art seeing "swap_pager_getswapspace(24): failed" etc. in dmesg.
>>> This is the memory usage a while after scrubbing, note the values for Me=
m/Free and Swap:
>>> ```
>>> % top | head -n 7
>>> last pid:  8112;  load averages:  1.82,  1.77,  1.73  up 6+01:37:42 10:5=
3:48
>>> 35 processes:  1 running, 34 sleeping
>>> CPU:  4.9% user,  0.0% nice,  4.2% system,  0.2% interrupt, 90.7% idle
>>> Mem: 110G Active, 27G Inact, 5413M Laundry, 39G Wired, 68G Free
>>> ARC: 34G Total, 28G MFU, 4101M MRU, 53M Anon, 1317M Header, 225M Other
>>>     30G Compressed, 53G Uncompressed, 1.77:1 Ratio
>>> Swap: 8192M Total, 6434M Used, 1757M Free, 78% Inuse
>>> ```
>>> We are running MySQL, which has been configured to use ~50% of the total=
 amount memory (using innodb_buffer_pool_size=3D127748M)
>>> ZFS ARC has been configured to use 25% of the total memory (using vfs.zf=
s.arc_max=3D"63874M")
>>> We have tried raising both vfs.zfs.arc_max and innodb_buffer_pool_size, b=
ut this did not make any change to the total memory usage, the free memory s=
tays at around 70G and FreeBSD still started swapping.
>>> It's as if the memory is capped at around 180G for some reason.
>>> Are there any configuration values that could cause FreeBSD to swap even=
 though there's free memory? Are there any config values one could try to ch=
ange in order to get FreeBSD to use the remaining ~70G of free memory instea=
d of swapping?
>>> Let me know if there's any more details you want me to provide and I'll a=
ttach those.
>>> Thanks!
>>> // Mattias
>=20
> I see similar things. The Jails in question are 11.1. The systems updated t=
o 12.1 do not display this behavior. This 11.1 system runs 5 jails. Swapinfo=
 is shown below:
>=20
> Device          1K-blocks     Used    Avail Capacity
> /dev/aacd0p3      4194304  1776000  2418304    42%
>=20
> These numbers are developed from top on the base system
>=20
> [ 0 50861 ]  root
> [ 20 281903 ]  camden         squirellmail/roundcube/postfix/mysql
> [ 21 322759 ]  bassharbor     wordpress/php56
> [ 19 343522 ]  monhegan       wordpress/php56
> [ 18 369139 ]  newharbor      apache24/sendmail
> [ 17 587332 ]  pemaquid       wordpress/php74
>=20
> Jails:  1904655
> total:  1955516
>=20
> I read somewhere that the virtual memory system pre-pages modified pages a=
s a just-in-case measure. If this is correct, 12.1 does not do this on a non=
-paging system. The system shown above uses about 10% swapspace after a rebo=
ot and works its way to the 42% shown above in a day or so.



Want to link to this message? Use this URL: <http://docs.FreeBSD.org/cgi/mid.cgi?05399132-C7A7-4B96-B6E4-D469CCDC70D5>