Date: Tue, 12 Feb 2019 13:42:56 -0500 From: Mark Johnston <markj@freebsd.org> To: Eugene Grosbein <eugen@grosbein.net> Cc: FreeBSD stable <freebsd-stable@freebsd.org> Subject: Re: 11.2-STABLE kernel wired memory leak Message-ID: <20190212184256.GC29847@raichu> In-Reply-To: <973d9dc6-8dd4-8cdf-b279-f5c9483da884@grosbein.net> References: <d8c7abc0-3ba1-40e4-22b1-1b30d28ced14@grosbein.net> <20190212163446.GA29847@raichu> <4ab1331f-80e3-b856-b402-9985e73618bc@grosbein.net> <20190212181805.GB29847@raichu> <973d9dc6-8dd4-8cdf-b279-f5c9483da884@grosbein.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, Feb 13, 2019 at 01:40:06AM +0700, Eugene Grosbein wrote: > 13.02.2019 1:18, Mark Johnston wrote: > > > On Wed, Feb 13, 2019 at 01:03:37AM +0700, Eugene Grosbein wrote: > >> 12.02.2019 23:34, Mark Johnston wrote: > >> > >>> I suspect that the "leaked" memory is simply being used to cache UMA > >>> items. Note that the values in the FREE column of vmstat -z output are > >>> quite large. The cached items are reclaimed only when the page daemon > >>> wakes up to reclaim memory; if there are no memory shortages, large > >>> amounts of memory may accumulate in UMA caches. In this case, the sum > >>> of the product of columns 2 and 5 gives a total of roughly 4GB cached. > >>> > >>>> as well as "sysctl hw": http://www.grosbein.net/freebsd/leak/sysctl-hw.txt > >>>> and "sysctl vm": http://www.grosbein.net/freebsd/leak/sysctl-vm.txt > >> > >> It seems page daemon is broken somehow as it did not reclaim several gigs of wired memory > >> despite of long period of vm thrashing: > > > > Depending on the system's workload, it is possible for the caches to > > grow quite quickly after a reclaim. If you are able to run kgdb on the > > kernel, you can find the time of the last reclaim by comparing the > > values of lowmem_uptime and time_uptime. > > Yes, I have debugger compiled into running kernel and have console access. > What commands should I use? I meant kgdb(1). If you can run that, try: (kgdb) p time_uptime (kgdb) p lowmem_uptime If you are willing to drop the system into DDB, do so and run db> x time_uptime db> x lowmem_uptime
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20190212184256.GC29847>