Date: Thu, 09 Jun 2005 21:15:36 +0200 From: "Poul-Henning Kamp" <phk@phk.freebsd.dk> To: Bakul Shah <bakul@BitBlocks.com> Cc: freebsd-fs@freebsd.org, =?iso-8859-1?q?Dag-Erling_Sm=F8rgrav?= <des@des.no>, FreeBSD Current <freebsd-current@freebsd.org> Subject: Re: you are in an fs with millions of small files Message-ID: <9019.1118344536@critter.freebsd.dk> In-Reply-To: Your message of "Thu, 09 Jun 2005 12:11:15 PDT." <200506091911.j59JBGQn096385@gate.bitblocks.com>
next in thread | previous in thread | raw e-mail | index | archive | help
In message <200506091911.j59JBGQn096385@gate.bitblocks.com>, Bakul Shah writes: >Start with a small array. When it gets full double it (or >grow by half if you want to waste less memory). If I might make an observation... You do not waste malloc'ed memory until you access it. You waste some page table entries etc in the kernel, but the actual pages do not become part of the memory-pressure mob-rule until you touch it the first time. This is a very important point which people still (after 20 years of virtual memory systems and 10 years of phkmalloc) still do not seem to have fully understood. I would advice a simple doubling and a terminal realloc to cut down to actual size. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?9019.1118344536>