Date: Tue, 24 Apr 2001 11:35:36 +0200 From: Niek Bergboer <niek@wit379119.student.utwente.nl> To: Mike Silbersack <silby@silby.com> Cc: freebsd-hackers@freebsd.org Subject: Re: UFS block size vs. write speed Message-ID: <20010424113536.A61988@wit379119.student.utwente.nl> In-Reply-To: <Pine.BSF.4.31.0104231207360.1931-100000@achilles.silby.com>; from silby@silby.com on Mon, Apr 23, 2001 at 12:09:26PM -0500 References: <20010423134554.A57241@wit379119.student.utwente.nl> <Pine.BSF.4.31.0104231207360.1931-100000@achilles.silby.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, Apr 23, 2001 at 12:09:26PM -0500, Mike Silbersack wrote:
> On Mon, 23 Apr 2001, Niek Bergboer wrote:
> > I've implemented a quick hack on the BSD ftp-client: in the original
> > recv-file function data is read from a socket into a buffer, which is
> > then written to a file. I've mmap-ed the file, and rather than reading
> > from the socket into the buffer, I read directly from the socket into
> > the mmaped region. I use the MAP_SHARED and MAP_NOSYNC flags, and
> > especially the latter makes a huge difference.
>
> I seem to remember Matt Dillon saying that if you extend a file through
> mmap'd writes, it will become very fragmented. Check the archives for
> this mailing list during decemeber or so, I believe the thread had
> something to so with usenet servers.
Yes, the file gets fragmented indeed because I first extend it using
ftruncate() and then mmap() it. The fragmentation gets especially bad
when the MAP_ASYNC flag is passed to mmap(). However, in order to get
a "write-cache"-like system I need to use MAP_ASYNC.
Now, what I'm wondering about is the following: when one uses
ftruncate() to extend a file, _no_ blocks get allocated in the
filesystem because all blocks are empty (it was nice to see that I was
actually able to make a 800 GB "file" on a 15 GB harddisk).
This non-allocation seems to be the problem: if more than one process
is writing, the blocks will have to be allocated dynamically as one
writes into the mmap()'ed area. So is it possible to _allocate_ the
blocks without actually _writing_ them full of zeros? If ftruncate()
can do this, you would only have to write a very small amount of
meta-data to the filesystem in order to allocate a non-fragmented set
of blocks.
In fact, I couldn't care less if the allocated blocks contain random
data (rather than zeros), since I'll be overwriting them immediately.
> Mike "Silby" Silbersack
Niek Bergboer
--
Conscience doth make cowards of us all.
-- Shakespeare
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20010424113536.A61988>