From owner-freebsd-current@FreeBSD.ORG Thu Jan 31 22:25:24 2013 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 423C022A; Thu, 31 Jan 2013 22:25:24 +0000 (UTC) (envelope-from ian@FreeBSD.org) Received: from duck.symmetricom.us (duck.symmetricom.us [206.168.13.214]) by mx1.freebsd.org (Postfix) with ESMTP id F2EFF7F4; Thu, 31 Jan 2013 22:25:23 +0000 (UTC) Received: from damnhippie.dyndns.org (daffy.symmetricom.us [206.168.13.218]) by duck.symmetricom.us (8.14.6/8.14.6) with ESMTP id r0VMPNPf015077; Thu, 31 Jan 2013 15:25:23 -0700 (MST) (envelope-from ian@FreeBSD.org) Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id r0VMPLIJ025445; Thu, 31 Jan 2013 15:25:21 -0700 (MST) (envelope-from ian@FreeBSD.org) Subject: Re: Trouble with recent auto-tuning changes From: Ian Lepore To: Andre Oppermann In-Reply-To: <510AA63D.2080301@freebsd.org> References: <1359310302.93359.48.camel@revolution.hippie.lan> <1359382968.93359.66.camel@revolution.hippie.lan> <5106CF85.8060004@rice.edu> <510AA63D.2080301@freebsd.org> Content-Type: text/plain; charset="us-ascii" Date: Thu, 31 Jan 2013 15:25:20 -0700 Message-ID: <1359671120.93359.343.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: alc@FreeBSD.org, freebsd-current@FreeBSD.org, Alan Cox X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Jan 2013 22:25:24 -0000 On Thu, 2013-01-31 at 18:13 +0100, Andre Oppermann wrote: > On 28.01.2013 20:20, Alan Cox wrote: > > On 01/28/2013 08:22, Ian Lepore wrote: > >> On Mon, 2013-01-28 at 00:09 -0600, Alan Cox wrote: > >>> On Sun, Jan 27, 2013 at 12:11 PM, Ian Lepore wrote: > >>> > >>>> I ran into a panic while attempting to un-tar a large file on a > >>>> DreamPlug (arm-based system) running -current. The source and dest of > >>>> the un-tar is the root filesystem on sdcard, and I get this: > >>>> > >>>> panic: kmem_malloc(4096): kmem_map too small: 12582912 total allocated > >>>> > >>>> Just before the panic I see the tar process get hung in a "nokva" wait. > >>>> 12582912 is the value of VM_KMEM_SIZE from arm/include/vmparam.h. > >>>> > >>>> In r245575 the init order for mbuf limits was changed from > >>>> SI_SUB_TUNABLES to SI_SUB_KMEM so that mbuf limits could be based on the > >>>> results of sizing kernel memory. Unfortunately, the process of sizing > >>>> kernel memory relies on the mbuf limits; in kmeminit(): > >>>> > >>>> vm_kmem_size = VM_KMEM_SIZE + nmbclusters * PAGE_SIZE; > >>>> > >>>> Since r245575, nmbclusters is zero when this line of code runs. If I > >>>> manually plugin "32768" (the number tunable_mbinit() comes up with for > >>>> this platform) in that line, the panic stops happening. > >>>> > >>>> So we've got two problems here... one is the circular dependency in > >>>> calculating the mbuf limits. The other is the fact that some > >>>> non-trivial amount of kernel memory we're allowing for mbufs is actually > >>>> being used for other things. That is, if my system was actually using > >>>> all the mbufs that tunable_mbinit() allowed for, then this panic while > >>>> untarring a huge file would still have happened. > >>>> > >>>> > >>> All of this is factually correct. However, it's a red herring. The real > >>> problem is that arm, unlike every other architecture in the tree, does not > >>> enable auto-sizing of the kmem map based on the physical memory size. > >>> Specifically, you'll find VM_KMEM_SIZE_SCALE defined in > >>> "arch"/include/vmparam.h on every other architecture, just not on arm. > >>> This auto-sizing overrides the value of VM_KMEM_SIZE. > >>> > >> Aha. I'll investigate what other architectures do with that and try to > >> get the same thing going for arm. > >> > > > > i386 or (32-bit) MIPS would be the most similar. Also, I would > > encourage you to look for other definitions that those architectures > > have that arm doesn't. As physical memory sizes continue to grow on > > arm-based systems, they may require other changes in vmparam.h and the > > machine-dependent param.h that were made on those other architectures > > year ago. > > Ian, > > The patch below should do the trick. Can you please test? Yep, that fixed the problem with untarring the large file. Here are some before/after numbers from sysctl, converted from bytes to KBytes for readability: vm.kmem_size_scale: 0 2 vm.kmem_map_free: 5740 246440 vm.kmem_map_size: 6548 7176 vm.kmem_size: 12288 253616 real memory = 536870912 (512 MB) avail memory = 516718592 (492 MB) -- Ian