From owner-freebsd-hackers Mon Apr 23 4:42:39 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from wit379112.student.utwente.nl (wit379119.student.utwente.nl [130.89.232.129]) by hub.freebsd.org (Postfix) with ESMTP id 3F0CB37B424 for ; Mon, 23 Apr 2001 04:42:36 -0700 (PDT) (envelope-from niek@wit379112.student.utwente.nl) Received: by wit379112.student.utwente.nl (Postfix, from userid 1000) id A496C5D49; Mon, 23 Apr 2001 13:45:54 +0200 (CEST) Date: Mon, 23 Apr 2001 13:45:54 +0200 From: Niek Bergboer To: Alfred Perlstein Cc: freebsd-hackers@freebsd.org Subject: Re: UFS block size vs. write speed Message-ID: <20010423134554.A57241@wit379119.student.utwente.nl> Reply-To: niek@bergboer.net References: <20010420144543.F30241@wit379119.student.utwente.nl> <20010420055426.Q1790@fw.wintelcom.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010420055426.Q1790@fw.wintelcom.net>; from bright@wintelcom.net on Fri, Apr 20, 2001 at 05:54:26AM -0700 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Fri, Apr 20, 2001 at 05:54:26AM -0700, Alfred Perlstein wrote: > > PS: The tests were already done with the fs mounted async. The > > drive in question communicates at UDMA/33 on a PIIX4 controller in > > an AMD K6/2 233 system. > It's funny, but you have the ideal system for an interesting > optimization I've always wanted to try. Since you seem to be > reading over the network, have you tried doing this, creating > the file and then using ftruncate on it to extend it, then use > mmap() and read directly from the socket into the mmap'd area. I've implemented a quick hack on the BSD ftp-client: in the original recv-file function data is read from a socket into a buffer, which is then written to a file. I've mmap-ed the file, and rather than reading from the socket into the buffer, I read directly from the socket into the mmaped region. I use the MAP_SHARED and MAP_NOSYNC flags, and especially the latter makes a huge difference. The files are read using FTP with a buffer size of 64 kb. They are read from a Linux 2.4 machine with cartloads of memory and I make sure the Linux box has the files cached in memory. The Linux machine is connected to the same 100 MBit/FDX switch and network congestion is virtually non-existant. Because the FreeBSD machine (the K6/2 233) only has 64 MB of RAM, I've made the files small (10 files of 4 MB each, and later 5 files of 8 MB each). The normal BSD ftp-client reads all the files a approx. 6.5 to 7 Mbyte/s and this speed is more or less constant for all the files. The mmap-ed version of the ftp-client reads the first few files at 8 to 8.5 MByte/s after which the throughput collapses due to the limited amount of memory available for caching. I've noticed that using madvise() with MADV_WILLNEED or MADV_SEQUENTIAL does not make that much of a difference. Conclusion: using the mmap trick, the box displays the ugly "all your system is one big write cache" Linux-behaviour ;) Until memory is full that is, after which throutput becomes lousy (4.5 Mbyte/s). > You may have to experiment with several different madvise() flags > to get optimal performance. Or you may discover that doing this > "trick" actually makes performance worse because of the way > the trick screws with what the vm system expects. > I think a combination of MADV_SEQUENTIAL and/or MADV_WILLNEED > could do the trick. See above. Niek -- Conscience doth make cowards of us all. -- Shakespeare To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message