From owner-freebsd-hackers Tue Apr 24 8: 7:50 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from silby.com (adam042-060.resnet.wisc.edu [146.151.42.60]) by hub.freebsd.org (Postfix) with ESMTP id 84C4D37B423 for ; Tue, 24 Apr 2001 08:07:46 -0700 (PDT) (envelope-from silby@silby.com) Received: (qmail 4393 invoked by uid 1000); 24 Apr 2001 15:07:42 -0000 Received: from localhost (sendmail-bs@127.0.0.1) by localhost with SMTP; 24 Apr 2001 15:07:42 -0000 Date: Tue, 24 Apr 2001 10:07:42 -0500 (CDT) From: Mike Silbersack To: Cc: Subject: Re: UFS block size vs. write speed In-Reply-To: <20010424113536.A61988@wit379119.student.utwente.nl> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Tue, 24 Apr 2001, Niek Bergboer wrote: > On Mon, Apr 23, 2001 at 12:09:26PM -0500, Mike Silbersack wrote: > > On Mon, 23 Apr 2001, Niek Bergboer wrote: > > > I've implemented a quick hack on the BSD ftp-client: in the original > > > recv-file function data is read from a socket into a buffer, which is > > > then written to a file. I've mmap-ed the file, and rather than reading > > > from the socket into the buffer, I read directly from the socket into > > > the mmaped region. I use the MAP_SHARED and MAP_NOSYNC flags, and > > > especially the latter makes a huge difference. > > > > I seem to remember Matt Dillon saying that if you extend a file through > > mmap'd writes, it will become very fragmented. Check the archives for > > this mailing list during decemeber or so, I believe the thread had > > something to so with usenet servers. > > Yes, the file gets fragmented indeed because I first extend it using > ftruncate() and then mmap() it. The fragmentation gets especially bad > when the MAP_ASYNC flag is passed to mmap(). However, in order to get > a "write-cache"-like system I need to use MAP_ASYNC. Hm... mmap sounds like it'd be the best performer here, but clearly fixing the fragmentation issue is beyond the scope of the ftp testing. Perhaps there's a way to improve the concurrency of the network and disk I/O without needing mmap, though. Have you tried making a large circular buffer, perhaps 256K in size or so, from which you use non-blocking socket I/O to fill it, and the aio_* functions to write to disk? It would probably be more work than adding the use of mmap was, but it would be interesting to see the results. Mike "Silby" Silbersack To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message