Date: Mon, 2 Jul 2007 02:19:31 +0400 From: "Nguyen Tam Chinh" <unixvn@gmail.com> To: "Nikolay Pavlov" <quetzal@zone3000.net>, "Nguyen Tam Chinh" <unixvn@gmail.com>, FreeBSD-Questions <freebsd-questions@freebsd.org>, freebsd-stable@freebsd.org Subject: Re: UFS2 optimization for many small files Message-ID: <64b284310707011519k68ebc35p716cd900d9212234@mail.gmail.com> In-Reply-To: <20070701215229.GB10527@zone3000.net> References: <64b284310706270311j2a6af2f6i6766b483a4b66a5c@mail.gmail.com> <20070701215229.GB10527@zone3000.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On 7/2/07, Nikolay Pavlov <quetzal@zone3000.net> wrote: > On Wednesday, 27 June 2007 at 14:11:19 +0400, Nguyen Tam Chinh wrote: > > Greetings, > > > > We're going to build a server with some 1Tb of over 500 million small > > files with size from 0,5k to 4k. I'm wonder if the ufs2 can handle > > this kind of system well. From newfs(8) the min block size is 4k. This > > is not optimal in our case, a 1k or 0,5k block is more effective IMHO. > > I'd be happy if anyone can suggest what does fragment (block/8) in the > > ufs2 mean and how this parameter works. I know It's better to read the > > full ufs2 specification, but hope that someone here can give a hint. > > Please advice with optimizations or tricks. > > Thank you very much. > > > > -- > > With best regards, | The Power to Serve > > Nguyen Tam Chinh | http://www.FreeBSD.org > > _______________________________________________ > > freebsd-stable@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > > I am not aware of any ZFS results on such tasks, may be you will be the > one who share them ;) However RaiserFS whould be the best choise on such > spesific case. It's not available on FreeBSD currently. > I don't think UFS can handle a huge amount of small files effectively. > Of course gjournal could be an option for fsck problems, but how do you > plan to backup or sync this storage? > I'm aware of the fsck/backup problems. In our case there's no need for backup so i went with ufs2. The current configuration is 4x250Gb disks with bloc/frag ratio 4k/512b. We're generating files with the average size of 6k ('cause the compress procedure does not work as well as we estimated). After a week I think we could collect some statistics in production. Anyway, in this case a 8k/1k would be more effective for us. Hope that I can test this in the next server. -- With best regards, | The Power to Serve Nguyen Tam Chinh | http://www.FreeBSD.org
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?64b284310707011519k68ebc35p716cd900d9212234>