Date: Sun, 26 Mar 2006 08:27:20 +1100 From: Peter Jeremy <peterjeremy@optushome.com.au> To: Mikhail Teterin <mi+kde@aldan.algebra.com> Cc: alc@freebsd.org, Mikhail Teterin <mi+mx@aldan.algebra.com>, stable@freebsd.org Subject: Re: Reading via mmap stinks (Re: weird bugs with mmap-ing via NFS) Message-ID: <20060325212720.GI703@turion.vk2pj.dyndns.org> In-Reply-To: <200603250920.14208@aldan> References: <200603232352.k2NNqPS8018729@gate.bitblocks.com> <200603241518.01027.mi%2Bmx@aldan.algebra.com> <20060325103927.GE703@turion.vk2pj.dyndns.org> <200603250920.14208@aldan>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sat, 2006-Mar-25 09:20:13 -0500, Mikhail Teterin wrote: >I'm sorry, that should be http://aldan.algebra.com/~mi/mzip.c -- I checked >this time :-( It doesn't look like it's doing anything especially weird. As Matt pointed out, creating files with mmap() is not a good idea because the syncer can cause massive fragmentation when allocating space. I can't test is as-is because it insists on mmap'ing its output and I only have one disk and you can't mmap /dev/null. Since your program is already written to mmap the input and output in pieces, it would be trivial to convert it to use read/write. >= I tried writing a program that just mmap'd my entire (2GB) test file >= and summed all the longwords in it. > >The files I'm dealing with are database dumps -- 10-80Gb :-) Maybe, that's, >what triggers some pessimal case?.. I tried generating an 11GB test file and got results consistent with my previous tests: grep using read or mmap, as well as mmap'ing the entire file give similar times with the disk mostly saturated. I suggest you try converting mzip.c to use read/write and see if the problem is still present. -- Peter Jeremy
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20060325212720.GI703>