Date: Fri, 16 Oct 2020 18:54:06 +0100 From: Jon Schneider <freebsd@jschneider.net> To: doug@safeport.com Cc: freebsd-questions@freebsd.org, Twingly Customer Support <team@twingly.com> Subject: Re: FreeBSD using swap even though there's a lot of free memory Message-ID: <05399132-C7A7-4B96-B6E4-D469CCDC70D5@jschneider.net>
next in thread | raw e-mail | index | archive | help
=EF=BB=BFSunOS 4 used to (speculatively) page stuff out. Probably just indic= ates relatively unused pages. Jon > On 16 Oct 2020, at 18:50, doug <doug@fledge.watson.org> wrote: >=20 > =EF=BB=BF >=20 >> On Thu, 15 Oct 2020, Jon Schneider wrote: >> top -w -oswap >> seems to report the right thing. Would be interesting if it's _not_ MySQL= . >> Jon >>> On 15/10/2020 15:23, Twingly Customer Support wrote: >>> Hi, >>> We have a server running FreeBSD 12.1-RELEASE-p10. We currently have a p= roblem where FreeBSD starting to swap when running ZFS scrub, even though we= have ~70G of free memory. >>> This did not happen before when running FreeBSD 11.3 for example. It sta= rted happening at approximately the time we upgraded from 12.1-RELEASE-p5 to= 12.1-RELEASE-p6, but if the upgrade is the cause of the problem is unclear,= though FreeBSD never swapped for us before that. "Laundry" memory was not s= omething we saw before either, it started to appear at the same time as Free= BSD started swapping. >>> Eventually, after scrubbing a few times, the swap becomes full and we st= art seeing "swap_pager_getswapspace(24): failed" etc. in dmesg. >>> This is the memory usage a while after scrubbing, note the values for Me= m/Free and Swap: >>> ``` >>> % top | head -n 7 >>> last pid: 8112; load averages: 1.82, 1.77, 1.73 up 6+01:37:42 10:5= 3:48 >>> 35 processes: 1 running, 34 sleeping >>> CPU: 4.9% user, 0.0% nice, 4.2% system, 0.2% interrupt, 90.7% idle >>> Mem: 110G Active, 27G Inact, 5413M Laundry, 39G Wired, 68G Free >>> ARC: 34G Total, 28G MFU, 4101M MRU, 53M Anon, 1317M Header, 225M Other >>> 30G Compressed, 53G Uncompressed, 1.77:1 Ratio >>> Swap: 8192M Total, 6434M Used, 1757M Free, 78% Inuse >>> ``` >>> We are running MySQL, which has been configured to use ~50% of the total= amount memory (using innodb_buffer_pool_size=3D127748M) >>> ZFS ARC has been configured to use 25% of the total memory (using vfs.zf= s.arc_max=3D"63874M") >>> We have tried raising both vfs.zfs.arc_max and innodb_buffer_pool_size, b= ut this did not make any change to the total memory usage, the free memory s= tays at around 70G and FreeBSD still started swapping. >>> It's as if the memory is capped at around 180G for some reason. >>> Are there any configuration values that could cause FreeBSD to swap even= though there's free memory? Are there any config values one could try to ch= ange in order to get FreeBSD to use the remaining ~70G of free memory instea= d of swapping? >>> Let me know if there's any more details you want me to provide and I'll a= ttach those. >>> Thanks! >>> // Mattias >=20 > I see similar things. The Jails in question are 11.1. The systems updated t= o 12.1 do not display this behavior. This 11.1 system runs 5 jails. Swapinfo= is shown below: >=20 > Device 1K-blocks Used Avail Capacity > /dev/aacd0p3 4194304 1776000 2418304 42% >=20 > These numbers are developed from top on the base system >=20 > [ 0 50861 ] root > [ 20 281903 ] camden squirellmail/roundcube/postfix/mysql > [ 21 322759 ] bassharbor wordpress/php56 > [ 19 343522 ] monhegan wordpress/php56 > [ 18 369139 ] newharbor apache24/sendmail > [ 17 587332 ] pemaquid wordpress/php74 >=20 > Jails: 1904655 > total: 1955516 >=20 > I read somewhere that the virtual memory system pre-pages modified pages a= s a just-in-case measure. If this is correct, 12.1 does not do this on a non= -paging system. The system shown above uses about 10% swapspace after a rebo= ot and works its way to the 42% shown above in a day or so.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?05399132-C7A7-4B96-B6E4-D469CCDC70D5>