From owner-freebsd-hackers Wed May 12 16:59:33 1999 Delivered-To: freebsd-hackers@freebsd.org Received: from allegro.lemis.com (allegro.lemis.com [192.109.197.134]) by hub.freebsd.org (Postfix) with ESMTP id 581EA15DD0 for ; Wed, 12 May 1999 16:59:23 -0700 (PDT) (envelope-from grog@freebie.lemis.com) Received: from freebie.lemis.com (freebie.lemis.com [192.109.197.137]) by allegro.lemis.com (8.9.1/8.9.0) with ESMTP id JAA03347; Thu, 13 May 1999 09:28:12 +0930 (CST) Received: (from grog@localhost) by freebie.lemis.com (8.9.3/8.9.0) id JAA15297; Thu, 13 May 1999 09:28:08 +0930 (CST) Date: Thu, 13 May 1999 09:28:08 +0930 From: Greg Lehey To: "Mark J. Taylor" Cc: Matthew Jacob , freebsd-hackers@FreeBSD.ORG, Jim Carroll Subject: Re: fsck and large file system Message-ID: <19990513092808.X89091@freebie.lemis.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.95.4i In-Reply-To: ; from Mark J. Taylor on Wed, May 12, 1999 at 03:18:22PM -0400 WWW-Home-Page: http://www.lemis.com/~grog X-PGP-Fingerprint: 6B 7B C3 8C 61 CD 54 AF 13 24 52 F8 6D A4 95 EF Organization: LEMIS, PO Box 460, Echunga SA 5153, Australia Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-41-739-7062 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Wednesday, 12 May 1999 at 15:18:22 -0400, Mark J. Taylor wrote: > On 12-May-99 Matthew Jacob wrote: >> >> >>>> I was wondering if anyone has done any work on fsck and very large file >>>> systems. We have a system that has 126 GB RAID Array. As you can imagine, >>>> fsck chokes trying to alloc enough blocks to store it's internal data >>>> structures (128 MB RAM, 128 MB Swap) >> >> Huh- I remember fixing this for NetBSD. You have to do a setrlimit within >> fsck so it can malloc enough space and have enough swap to back that. We >> were fsck'ing 600GB+ filesystems. >> >>> >>>> We would like to treat this array as a single large disk, and was wondering >>>> if anyone else had run into this situation, and had a work around. >>> >> >> I've been doing 120GB+ filesystems for FreeBSD for quite some time. The >> real fun will be the 1TB filesystems. > > The problem that we ran into in a system with several 130 MB RAID5 arrays > is that the fsck was running out of RAM+swap. We had to add a vnode to swap > to before the fsck would complete (basically added more swap space). > We had to have over 100 MB swap space to fsck the 130 MB volume, and the > system has 64 MB RAM. This was is 2.2.8 (haven't upgraded it yet). Why can't you just use more swap rather than resort to vnodes? > BTW: this system is getting VERY poor I/O performance, using the DPT SCSI RAID > controller and three arrays of four 49 GB Seagate drives. "iozone" reports > 340,000 bytes/sec write and 9,800,000 bytes/sec read. That's particularly bad. Which model was it? I've done some comparisons with Vinum and found that the write performance of Vinum (RAID-5) was round 25% of read performance, while it was below 10% on the SmartRAID IV. You should also try rawio (ftp://ftp.lemis.com/pub/rawio.tar.gz). iozone and bonnie use block devices, and they measure the total system, not just the storage device. > This horrendous write rate makes the system virtually unusable. > Anyone have any ideas on improving the performance? Use Vinum :-) > Would an upgrade from its 2.2.8 to 3.{1,2} help? It doesn't seem to be a FreeBSD issue. > It is a Pentium 166. During the "iozone" test, there seems to be > only a few (less than 10) interrupts from the DPT card per second > ("systat -vm 1"). Am I losing interrupts (it would seem so)... ? I think you'd get a message if you lost interrupts. It would be interesting to correlate the interrupt load with the test you're doing. Greg -- See complete headers for address, home page and phone numbers finger grog@lemis.com for PGP public key To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message