Date: Wed, 24 May 2017 12:07:13 +0300 From: Konstantin Belousov <kostikbel@gmail.com> To: Shrikanth Kamath <shrikanth07@gmail.com> Cc: freebsd-hackers@freebsd.org Subject: Re: Difference between p_vmspace quota between stable/11 and stable/10 Message-ID: <20170524090713.GG1622@kib.kiev.ua> In-Reply-To: <CAEOAkMWuptktG0UmNUuePwm6UZKkghyLUQoJtqzF4=wGoCYXSA@mail.gmail.com> References: <CAEOAkMWuptktG0UmNUuePwm6UZKkghyLUQoJtqzF4=wGoCYXSA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, May 24, 2017 at 01:00:51AM -0700, Shrikanth Kamath wrote: > I have a certain application(32 bit user space running in compat mode > in 64 bit system X86) which does a fork, and the child process does a > series of mmap calls as part of a scaling test. I am currently > debugging an issue with this application which hits a ENOMEM when it > is run on a stable/11 based setup and fails subsequent mmap and/or any > malloc calls v/s stable/10 where the issue is not seen.. I probed the > vm_map_find function using DTrace when "execname" was my application > in question, and got these readings > > fbt:kernel:vm_map_find:entry > /self->trace == 1/ /*enabled only during sys_mmap call of this application */ > { > @bytes[args[4]] = sum(args[4]); > printf("request length [%x]", args[4]); > } > > For stable_10 --> Total of 124 requests (length requested was > 0x500000) with the test successful > 124 * 0x500000 (5MB) ~ 620MB > > For stable_11 --> Total of 109 mmap requests > (0x500000/0x200000/0x3ff000 are the different vm_size_t length > arguments in vm_map_find). The test fails after 386MB has been > approved. > 24 * 0x500000 (5MB) ~ 120MB > 82 * 0x200000 (2MB) ~ 164MB > 3 * 0x3ff000 (4MB) ~ 12MB > > > The process parent rlimits are > > # cat /proc/5058/rlimit > > cpu -1 -1 > fsize -1 -1 > data 3221225472 3221225472 > stack 16777216 16777216 > core -1 -1 > rss 67108864 33265819648 > memlock 67108864 33265819648 > nproc 512 512 > nofile 1024 1024 > sbsize -1 -1 > vmem -1 -1 > npts -1 -1 > swap -1 -1 > kqueues -1 -1 > umtx -1 -1 > > The requests started failing in stable/11 with just 386 MB approved > v/s stable/10 which was successful in approving ~620MB. > > My stable/11 is from early days and is at GRN 302410 (probably 10 months old) > Any pointers or tips on what to probe further will be very helpful. Is > there any limits breach that I should probe further? The limits set > when a process is forked? > Should I probe the p->vmspace initiazliation? I doubt that limits are relevant for your issue. Look at the process address map at the moment when the request failed, I suspect that it is fragmented. Use procstat -v <pid> to examine the address space. You may spawn the tool from your program when mmap(2) fails.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20170524090713.GG1622>