From owner-freebsd-current@FreeBSD.ORG Mon Jun 21 00:18:12 2004 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5F29416A4CE; Mon, 21 Jun 2004 00:18:12 +0000 (GMT) Received: from rwcrmhc12.comcast.net (rwcrmhc12.comcast.net [216.148.227.85]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4C80043D5C; Mon, 21 Jun 2004 00:18:12 +0000 (GMT) (envelope-from julian@elischer.org) Received: from interjet.elischer.org ([24.7.73.28]) by comcast.net (rwcrmhc12) with ESMTP id <2004062100165601400211soe>; Mon, 21 Jun 2004 00:16:57 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id RAA26821; Sun, 20 Jun 2004 17:16:55 -0700 (PDT) Date: Sun, 20 Jun 2004 17:16:54 -0700 (PDT) From: Julian Elischer To: Matthew Dillon In-Reply-To: <200406210012.i5L0CJd6031533@apollo.backplane.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: questions@freebsd.org cc: current@freebsd.org cc: Mikhail Teterin Subject: Re: read vs. mmap (or io vs. page faults) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jun 2004 00:18:12 -0000 On Sun, 20 Jun 2004, Matthew Dillon wrote: > Hmm. Well, you can try calling madvise(... MADV_WILLNEED), that's what > it is for. > > It is usually a bad idea to try to populate the page table with all > resident pages associated with the a memory mapping, because mmap() > is often used to map huge files... hundreds of megabytes or even > dozens of gigabytes (on 64 bit architectures). The last thing you want > to do is to populate the page table for the entire file. It might > work for your particular program, but it is a bad idea for the OS to > assume that for every mmap(). > > What it comes down to, really, is whether you feel you actually need the > additional performance, because it kinda sounds to me that whatever > processing you are doing to the data is either going to be I/O bound, > or it isn't going to run long enough for the additional overhead to matter > verses the processing overhead of the program itself. > > If you are really worried you could pre-fault the mmap before you do > any processing at all and measure the time it takes to pre-fault the > pages vs the time it takes to process the memory image. (You pre-fault > simply by accessing one byte of data in each page across the mmap(), > before you begin any processing). pre-faulting is best done by a worker thread or child process, or it will just slow you down.. > > -Matt > Matthew Dillon > > > := It's hard to say. mmap() could certainly be made more efficient, e.g. > := by faulting in more pages at a time to reduce the actual fault rate. > := But it's fairly difficult to beat a read copy into a small buffer. > : > :Well, that's the thing -- by mmap-ing the whole file at once (and by > :madvise-ing with MADV_SEQUENTIONAL), I thought, I told, the kernel > :everything it needed to know to make the best decision. Why can't > :page-faulting code do a better job using all this knowledge, than the > :poor read, which only knows about the partial read in question? > : > :I find it so disappointing, that it can, probably, be considered a bug. > :I'll try this code on Linux and Solaris. If mmap is better there (as it > :really ought to be), we have a problem, IMHO. Thanks! > : > : -mi > : > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" >