Date: Wed, 24 Oct 2018 10:19:20 -0700 From: Robert <robert.ayrapetyan@gmail.com> To: Mark Johnston <markj@freebsd.org> Cc: freebsd-hackers@freebsd.org Subject: Re: Sudden grow of memory in "Laundry" state Message-ID: <e705099c-ea42-4985-1012-50e9fa11addd@gmail.com> In-Reply-To: <20180928152550.GA3609@raichu> References: <55b0dd7d-19a3-b566-0602-762b783e8ff3@gmail.com> <20180911005411.GF2849@raichu> <ce38cbfa-e1c5-776e-ef2e-2b867c9a520f@gmail.com> <20180911150849.GD92634@raichu> <104be96a-c16b-7e7c-7d0d-00338ab5a106@gmail.com> <20180928152550.GA3609@raichu>
next in thread | previous in thread | raw e-mail | index | archive | help
So the issue is still happening. Please check attached screenshot. The
green area is "inactive + cached + free".
At some point massive amount of "Free" memory is moving to Swap and
Laundry. Then slowly returns back to inactive.
My concern is that I've tried all kind of monitoring for mem
allocations, including dtrace-ing of all mmap calls.
It shows no any allocations of such a huge size, so I believe this is
something related to the kernel's mem management.
Could you advice which other places should be checked\monitored to find
a root cause?
Thanks.
On 9/28/18 6:25 AM, Mark Johnston wrote:
> On Thu, Sep 27, 2018 at 04:04:15PM -0700, Robert wrote:
>> Is there a way to force move pages back from laundry to Free or Inactive?
> There's no real way to do that from outside of the application. The
> application itself can free anonymous memory by unmapping it. It can
> also force memory to be marked clean (thus moving it back to the
> inactive queue) using madvise(MADV_FREE). This will cause the memory's
> contents to be discarded, though.
>
>> Also, what's the best way to identify addresses of these pages and
>> "look" inside of them?
> There's no convenient way to do that that I'm aware of. On amd64, you
> could in principle use kgdb to dump individual pages in the queue via
> the direct map:
>
> # kgdb82 -q
> Reading symbols from /boot/kernel/kernel...Reading symbols from /usr/lib/debug//boot/kernel/kernel.debug...done.
> done.
> sched_switch (td=0xfffff8002c560000, newtd=0xfffff800025ca000, flags=<optimized out>) at /home/mark/src/freebsd-dev/sys/kern/sched_ule.c:2112
> 2112 cpuid = PCPU_GET(cpuid);
> (kgdb) p vm_dom[0].vmd_pagequeues[2]
> $1 = {
> pq_mutex = {
> lock_object = {
> lo_name = 0xffffffff80c34c0d "vm laundry pagequeue",
> lo_flags = 21168128,
> lo_data = 0,
> lo_witness = 0xfffff8041ed6cb00
> },
> mtx_lock = 0
> },
> pq_pl = {
> tqh_first = 0xfffff8040f9ef980,
> tqh_last = 0xfffff8041227f448
> },
> pq_cnt = 20087,
> pq_name = 0xffffffff80c34c0d "vm laundry pagequeue",
> pq_pdpages = 0
> }
> (kgdb) x/512gx vm_dom[0].vmd_pagequeues[2].pq_pl.tqh_first->phys_addr + (vm_offset_t)&dmapbase
> 0xfffff801aea00000: 0x0000000800000000 0x000000082298c400
> 0xfffff801aea00010: 0x00000009be801000 0x0000000000000006
> ...
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?e705099c-ea42-4985-1012-50e9fa11addd>