Date: Mon, 24 Dec 2007 12:12:08 -0500 From: Scott Long <scottl@samsco.org> To: Brett Glass <brett@lariat.net> Cc: stable@freebsd.org Subject: Re: SMP on FreeBSD 6.x and 7.0: Worth doing? Message-ID: <476FE868.8080704@samsco.org> In-Reply-To: <200712241653.JAA20845@lariat.net> References: <200712220531.WAA09277@lariat.net> <476FBED0.2080400@samsco.org> <200712241549.IAA19650@lariat.net> <476FDA10.4060107@samsco.org> <200712241653.JAA20845@lariat.net>
next in thread | previous in thread | raw e-mail | index | archive | help
Brett Glass wrote: > At 09:10 AM 12/24/2007, Scott Long wrote: > >> Did you also nuke malloc debugging? > > I believe I did. I tried to take out all debugging to make it a fair test. > >>> They were. The drives are SATA. >> Connected to what controller? > > Whatever comes standard on the Intel S5000 motherboards. I believe that it > is Intel's own embedded SATA controller, which according to the spec sheet > is called their "ESB2-E" embedded SATA RAID controller. We are not using the > RAID capability. > >>>> 3. Directory hashing. If you're using squid, you __must__ tune the DIRHASH, otherwise you'll spend a lot of time doing pathname lookups. What filesystem is linux using? >>> Whatever comes standard with Ubuntu. >> Which is??? > > EXT3. > >>> My guess, based on what I saw, is that UFS2 doesn't take as much advantage of >>> SMP as Linux's file system does and threads are blocking on file I/O. >> That's really just speculation on your part, though. > > Yes, it is -- though I'd like to think it is an educated guess. ;-) > I'd need to instrument either the benchmark code or the kernel to > tell for sure. But the test is, by its nature, bound by file I/O. > >> I can tell you from my experience that a thrashed namei cache looks a >> lot like slow disk I/O to the casual observer, and that tuning the dirhash is highly important. > > That's always been true. However, Squid's file layout tends to be > gentle on the file system. It lays out directories with no more than > 256 items in each, and the names of the files are just sequential > hexadecimal numbers -- which I'd expect ought to bring about near-perfect > if not perfect hashing. It's not the same kind of hashing. The kind of "hashing" that squid does on the filesystem is sub optimal for UFS performance. > If you set the kernel to expect 256 or more entries > per directory (I know that Adrian Chadd is on this list; do you recommend > 256 or 512?), you seem to get the best performance you are going to get. It sounds like you're pretty convinced you know what the problem is. For others who might want help with this, tweaking vfs.ufs.dirhash_maxmem is what is needed. A bit of a balancing act is needed if you're on i386 since you'll risk exhausting KVA unless you also tweak KVA_PAGES. Scott
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?476FE868.8080704>