From owner-freebsd-questions@freebsd.org Fri Oct 16 20:24:21 2020 Return-Path: Delivered-To: freebsd-questions@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9C9E043F281 for ; Fri, 16 Oct 2020 20:24:21 +0000 (UTC) (envelope-from SRS0=mlVd=DX=mail.sermon-archive.info=doug@sermon-archive.info) Received: from mail.sermon-archive.info (sermon-archive.info [71.177.216.148]) by mx1.freebsd.org (Postfix) with ESMTP id 4CCd0h4T54z47ck for ; Fri, 16 Oct 2020 20:24:20 +0000 (UTC) (envelope-from SRS0=mlVd=DX=mail.sermon-archive.info=doug@sermon-archive.info) Received: from [10.0.1.251] (mini [10.0.1.251]) by mail.sermon-archive.info (Postfix) with ESMTPSA id 4CCd0Y6fWhz2fjWD for ; Fri, 16 Oct 2020 13:24:13 -0700 (PDT) From: Doug Hardie Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.17\)) Subject: Re: FreeBSD using swap even though there's a lot of free memory Date: Fri, 16 Oct 2020 13:24:13 -0700 References: <5f885b772d622_95aa2adab2b9c5b41576495c3@sirportly-app-02.mail> <20201016195546.a3392f971837aa2eeace1325@sohara.org> To: FreeBSD Mailing List In-Reply-To: <20201016195546.a3392f971837aa2eeace1325@sohara.org> Message-Id: <89B99A9E-C3A2-47BE-8A0E-D296846946CB@mail.sermon-archive.info> X-Mailer: Apple Mail (2.3445.104.17) X-Virus-Scanned: clamav-milter 0.101.4 at mail X-Virus-Status: Clean X-Rspamd-Queue-Id: 4CCd0h4T54z47ck X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of SRS0=mlVd=DX=mail.sermon-archive.info=doug@sermon-archive.info designates 71.177.216.148 as permitted sender) smtp.mailfrom=SRS0=mlVd=DX=mail.sermon-archive.info=doug@sermon-archive.info X-Spamd-Result: default: False [-1.35 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.91)[-0.915]; FROM_HAS_DN(0.00)[]; MV_CASE(0.50)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-questions@freebsd.org]; DMARC_NA(0.00)[lafn.org: no valid DMARC record]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-0.94)[-0.945]; TO_DN_ALL(0.00)[]; NEURAL_HAM_SHORT(-0.09)[-0.093]; R_SPF_ALLOW(-0.20)[+ip4:71.177.216.148]; FORGED_SENDER(0.30)[bc979@lafn.org,SRS0=mlVd=DX=mail.sermon-archive.info=doug@sermon-archive.info]; RCVD_NO_TLS_LAST(0.10)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:5650, ipnet:71.177.216.0/23, country:US]; FROM_NEQ_ENVFROM(0.00)[bc979@lafn.org,SRS0=mlVd=DX=mail.sermon-archive.info=doug@sermon-archive.info]; MAILMAN_DEST(0.00)[freebsd-questions]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Oct 2020 20:24:21 -0000 > On 16 October 2020, at 11:55, Steve O'Hara-Smith = wrote: >=20 > On Fri, 16 Oct 2020 11:35:42 -0700 > Michael Sierchio wrote: >=20 >> On Thu, Oct 15, 2020 at 7:24 AM Twingly Customer Support >> wrote: >>=20 >>> Hi, >>>=20 >>> We have a server running FreeBSD 12.1-RELEASE-p10. We currently have = a >>> problem where FreeBSD starting to swap when running ZFS scrub, even >>> though we have ~70G of free memory. >>=20 >>=20 >> tl;dr =E2=80=93 this is normal, nothing to see, move along. >=20 > Not when it starts running out of swap space while still = reporting > 70G of free memory. I encountered this issue a year or so ago. In my case it turned out to = be a process that was allocating anonymous segments using mmap. It = tended to forget to free them after the usage was complete. As a = result, the system would run out of swap even though there was plenty of = free memory. Anonymous segments are mapped to swap space. I traced the = problem using: procstat -va | grep df That generated a lot of data. I looked for processes that had an = increasing number of df files. Eventually I found the right one and = fixed it. Interesting observation: I have used FreeBSD since version 2.5 and for = years it always had a small number of processes swapped out within an = hour of a boot. Typically they were gettys and similar processes that = never got used. However, with 12.x, I now find that swap info is always = showing 0 blocks used even after weeks of operation. Something has = changed for the better. It is also possible that greping for sw rather than df might give some = useful information. I can't tell as none of my systems are using any = swap. Actually, one system did not have any swap for over a year. I = didn't notice that there was an issue mounting its swap. There were no = performance indicators that issue existed. Nothing changed after I = corrected the problem and added the swap. -- Doug