Date: Tue, 24 Apr 2001 10:07:42 -0500 (CDT) From: Mike Silbersack <silby@silby.com> To: <niek@bergboer.net> Cc: <freebsd-hackers@freebsd.org> Subject: Re: UFS block size vs. write speed Message-ID: <Pine.BSF.4.31.0104241001050.4372-100000@achilles.silby.com> In-Reply-To: <20010424113536.A61988@wit379119.student.utwente.nl>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, 24 Apr 2001, Niek Bergboer wrote: > On Mon, Apr 23, 2001 at 12:09:26PM -0500, Mike Silbersack wrote: > > On Mon, 23 Apr 2001, Niek Bergboer wrote: > > > I've implemented a quick hack on the BSD ftp-client: in the original > > > recv-file function data is read from a socket into a buffer, which is > > > then written to a file. I've mmap-ed the file, and rather than reading > > > from the socket into the buffer, I read directly from the socket into > > > the mmaped region. I use the MAP_SHARED and MAP_NOSYNC flags, and > > > especially the latter makes a huge difference. > > > > I seem to remember Matt Dillon saying that if you extend a file through > > mmap'd writes, it will become very fragmented. Check the archives for > > this mailing list during decemeber or so, I believe the thread had > > something to so with usenet servers. > > Yes, the file gets fragmented indeed because I first extend it using > ftruncate() and then mmap() it. The fragmentation gets especially bad > when the MAP_ASYNC flag is passed to mmap(). However, in order to get > a "write-cache"-like system I need to use MAP_ASYNC. Hm... mmap sounds like it'd be the best performer here, but clearly fixing the fragmentation issue is beyond the scope of the ftp testing. Perhaps there's a way to improve the concurrency of the network and disk I/O without needing mmap, though. Have you tried making a large circular buffer, perhaps 256K in size or so, from which you use non-blocking socket I/O to fill it, and the aio_* functions to write to disk? It would probably be more work than adding the use of mmap was, but it would be interesting to see the results. Mike "Silby" Silbersack To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.BSF.4.31.0104241001050.4372-100000>