Date: Fri, 21 Oct 2011 19:16:27 -0700 From: Doug Barton <dougb@FreeBSD.org> To: Miroslav Lachman <000.fbsd@quip.cz> Cc: freebsd-fs@freebsd.org, Ivan Voras <ivoras@freebsd.org> Subject: Re: dirhash and dynamic memory allocation Message-ID: <4EA2277B.5080306@FreeBSD.org> In-Reply-To: <4EA19203.5050503@quip.cz> References: <4E97FEDD.7060205@quip.cz> <j7938v$66s$1@dough.gmane.org> <4EA19203.5050503@quip.cz>
next in thread | previous in thread | raw e-mail | index | archive | help
On 10/21/2011 08:38, Miroslav Lachman wrote: > Hi, I am back on this topic... > > Ivan Voras wrote: >> On 14/10/2011 11:20, Miroslav Lachman wrote: >>> Hi all, >>> >>> I tried some tuning of dirhash on our servers and after googlig a bit, I >>> found an old GSoC project wiki page about Dynamic Memory Allocation for >>> Dirhash: http://wiki.freebsd.org/DirhashDynamicMemory >>> Is there any reason not to use it / not commit it to HEAD? >> >> AFAIK it's sort-of already present. In 8-stable and recent kernels you >> can give huge amounts of memory to dirhash via vfs.ufs.dirhash_maxmem >> (but except in really large edge cases I don't think you *need* more >> than 32 MB), and the kernel will scale-down or free the memory if not >> needed. >> >> In effect, vfs.ufs.dirhash_maxmem is the upper limit - the kernel will >> use less and will free the allocated memory in low memory situations >> (which I've tried and it works). > > So the current behavior is that on 7.3+ and 8.x we have smaller average > dirhash buffer (by default) than it was initialy 10 years ago. Because > it starts as 2MB fixed size and now we have 2MB max, which is lowered in > low mem situations... and sometimes it is set to 0MB! > > I caught this 2 days ago: > > root@rip ~/# sysctl vfs.ufs > vfs.ufs.dirhash_reclaimage: 5 > vfs.ufs.dirhash_lowmemcount: 36953 > vfs.ufs.dirhash_docheck: 0 > vfs.ufs.dirhash_mem: 0 > vfs.ufs.dirhash_maxmem: 8388608 > vfs.ufs.dirhash_minsize: 2560 > > I set maxmem to 8MB in sysctl.conf to increase performance and > dirhash_mem 0 is really bad surprise! > > I am worrying about bad performance in situation where dirhash is > emptied in situations, where server is already running at maximum > performance (there is some memory hungry process and system can start > swapping to disk + dirhash is efectively disabled) > > I found a PR kern/145246 > http://www.freebsd.org/cgi/query-pr.cgi?pr=145246 > > Is it possible to add some dirhash_minmem limit to not clear all the > dirhash memory? > So I can set dirhash_minmem=2MB dirhash_maxmem=16MB and then dirhash_mem > will be allways between these two limits? Isn't that what vfs.ufs.dirhash_minsize is for? I think given that there is a lot more memory in modern systems setting that higher by default is probably a good idea. Or maybe I'm misunderstanding what that knob does? >>> And second question - is there any negative impact with higher >>> vfs.ufs.dirhash_maxmem? It stil defaults to 2MB (on FreeBSD 8.2) after >> >> Not that I know of. >> >>> 10 years, but I think we all are using bigger FS in these days with lot >>> of files and directories and 2MB is not enough. >> >> AFAIK I've changed it to autotune so it's configured to approximately 4 >> MB on a 4 GB machine (and scales up) in 9. > > I didn't tried 9 yet. Does it mean dirhash_maxmem is initially set to > approximately 1% of physical RAM and then it can be set higher by sysctl > as in older versions? I'm not sure that's what's happening, I have 6G of ram in this box and I have this by default: vfs.ufs.dirhash_maxmem: 9977856 -- Nothin' ever doesn't change, but nothin' changes much. -- OK Go Breadth of IT experience, and depth of knowledge in the DNS. Yours for the right price. :) http://SupersetSolutions.com/
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4EA2277B.5080306>