Date: Mon, 19 Apr 1999 22:19:19 +0200 (CEST) From: Arjan de Vet <Arjan.deVet@adv.iae.nl> To: hackers@freebsd.org Subject: Re: Directories not VMIO cached at all! Message-ID: <199904192019.WAA05340@adv.iae.nl> In-Reply-To: <199904191650.JAA24137@vashon.polstra.com> References: <199904171844.LAA75452@apollo.backplane.com>
next in thread | previous in thread | raw e-mail | index | archive | help
In article <199904191650.JAA24137@vashon.polstra.com> John Polstra writes: >If I understand your description correctly, this fix could really >benefit master CVSup servers such as freefall. Those servers >typically have 8-12 running cvsupd processes, all doing tree walks >over the same CVS repository and making a stat() call on every file. > >Do you think it would help? It helps to some extend I think. The Squid server I've been testing can keep 32MB worth of directories cached after some tuning but because of the enormous amount of reads and writes being done half of the directories get removed from the cache after 5-10 minutes. We're speaking about 750,000-1,000,000 files in 4096 directories in a two-level hierarchy where 80% of the files is 9 KB or less in size. B.t.w., the Squid people are working on a SquidFS which will not use individual files anymore. Given the complete random read behavior of a proxy cache this means that for 50% of the cache-hits the directories and probably inodes too need to be fetched from disk before the real data can be read. And there's also a process 'unlinkd' which unlinks()s expired files (as far as I know also in a random way too, 16-32 files/sec). All this hurts performance of course. What is needed I think is a (preferably) sysctl variable which would favor metadata w.r.t. caching whenever possible. Matt's patch makes directories VMIO data too, so I don't know whether we still can decide easily whether a cached block is metadata or not (I just started studying the buffer cache code this weekend and it's quite complicated). Arjan To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199904192019.WAA05340>