Date: Tue, 11 Jun 2013 12:17:02 +0200 From: Attilio Rao <attilio@freebsd.org> To: Konstantin Belousov <kostikbel@gmail.com> Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org, Olivier Houchard <cognet@ci0.org>, Alan Cox <alc@rice.edu> Subject: Re: svn commit: r251586 - head/sys/arm/ti Message-ID: <CAJ-FndAie=PcrBk1jM33gPA9amDySy50EacJcaau2RvzT-jJGg@mail.gmail.com> In-Reply-To: <20130611052152.GG3047@kib.kiev.ua> References: <201306092251.r59MpCmW006162@svn.freebsd.org> <20130610035547.GX3047@kib.kiev.ua> <20130610110847.GA46614@ci0.org> <51B6069C.6060704@rice.edu> <20130610193736.GF3047@kib.kiev.ua> <20130610211358.GA55399@ci0.org> <20130610231052.GA57152@ci0.org> <20130611052152.GG3047@kib.kiev.ua>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, Jun 11, 2013 at 7:21 AM, Konstantin Belousov <kostikbel@gmail.com> wrote: > On Tue, Jun 11, 2013 at 01:10:52AM +0200, Olivier Houchard wrote: >> On Mon, Jun 10, 2013 at 11:13:58PM +0200, Olivier Houchard wrote: >> > On Mon, Jun 10, 2013 at 10:37:36PM +0300, Konstantin Belousov wrote: >> > > On Mon, Jun 10, 2013 at 12:02:20PM -0500, Alan Cox wrote: >> > > > On 06/10/2013 06:08, Olivier Houchard wrote: >> > > > > On Mon, Jun 10, 2013 at 06:55:47AM +0300, Konstantin Belousov wrote: >> > > > >> On Sun, Jun 09, 2013 at 10:51:12PM +0000, Olivier Houchard wrote: >> > > > >>> Author: cognet >> > > > >>> Date: Sun Jun 9 22:51:11 2013 >> > > > >>> New Revision: 251586 >> > > > >>> URL: http://svnweb.freebsd.org/changeset/base/251586 >> > > > >>> >> > > > >>> Log: >> > > > >>> Increase the maximum KVM available on TI chips. Not sure why we suddenly need >> > > > >>> that much, but that lets me boot with 1GB of RAM. >> > > > >> I suspect that the cause is the combination of limited KVA and >> > > > >> lack of any limitation for the buffer map. I noted that ARM lacks >> > > > >> VM_BCACHE_SIZE_MAX after a report from mav about similar (?) problem a >> > > > >> day ago. >> > > > >> >> > > > >> In essence, the buffer map is allowed to take up to ~330MB when no >> > > > >> upper limit from VM_BCACHE_SIZE_MAX is specified. >> > > > > >> > > > > Hi Konstantin, >> > > > > >> > > > > Thanks for the hint ! >> > > > > It seems only i386 and sparc64 sets it, what would be a good value, 200M, as >> > > > > it is on i386 ? >> > > > > >> > > > >> > > > Since there are many arm platforms with less than 1 GB of kernel virtual >> > > > address (KVA) space, VM_BCACHE_SIZE_MAX should be made to scale down >> > > > from 200 MB with the available KVA space. See how VM_KMEM_SIZE_MAX is >> > > > currently defined on arm. >> > > >> > > In fact, Ithink it does not make much sense to scale the buffer cache up. >> > > It is mostly wasted space now. As I measured it, on typical load you >> > > have only 10-20% of instantiated buffers mapped. >> > > >> > > Alexander Motin reported that he tested the equivalent of the following >> > > change. With it committed, I think that r251586 could be reverted. >> > > >> > > diff --git a/sys/arm/include/param.h b/sys/arm/include/param.h >> > > index 9ffb118..5c738c2 100644 >> > > --- a/sys/arm/include/param.h >> > > +++ b/sys/arm/include/param.h >> > > @@ -128,6 +128,11 @@ >> > > #define USPACE_SVC_STACK_BOTTOM (USPACE_SVC_STACK_TOP - 0x1000) >> > > #define USPACE_UNDEF_STACK_TOP (USPACE_SVC_STACK_BOTTOM - 0x10) >> > > #define USPACE_UNDEF_STACK_BOTTOM (FPCONTEXTSIZE + 10) >> > > + >> > > +#ifndef VM_BCACHE_SIZE_MAX >> > > +#define VM_BCACHE_SIZE_MAX (128 * 1024 * 1024) >> > > +#endif >> > > + >> > > /* >> > > * Mach derived conversion macros >> > > */ >> > >> > >> > I tested it with my changes reverted and it works indeed, so I'm fine with >> > this being committed and my changes being reverted. >> > >> >> In fact I spoke too soon. It's getting further, but I'm ending up getting >> vm_thread_new: kstack allocation failed >> Probably because I have a local patch that aligns the stack on 32kB, which >> is something we have to do if we want to store curthread on the kstack. >> It will boot if I reduce VM_DCACHE_SIZE_MAX to 64MB, but it's probably not >> the best thing to do. > > The other cause of increased KVA use is the vm radix trie used to keep > the collection of the vm object' pages. When I profiled KVA use for PAE > on i386, which has similar problem of exhausted KVA, the radix trie > popped up as the reason. BTW, it would be interesting to see data from your analysis. When radix trie was developed I get pho@ to test PAE too and no exhaustation was reported. Also, on a related note, I wonder how relevant is PAE support on i386 nowadays. Maybe we should consider axing it. Attilio -- Peace can only be achieved by understanding - A. Einstein
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAJ-FndAie=PcrBk1jM33gPA9amDySy50EacJcaau2RvzT-jJGg>