Date: Tue, 10 Jan 2012 17:49:01 -0600 (CST) From: Bob Friesenhahn <bfriesen@simple.dallas.tx.us> To: "Mikhail T." <mi+thun@aldan.algebra.com> Cc: fs@freebsd.org Subject: Re: How to best send files over network? Message-ID: <alpine.GSO.2.01.1201101741080.20764@freddy.simplesystems.org> In-Reply-To: <4F0CBB79.7010704@aldan.algebra.com> References: <4F0CBB79.7010704@aldan.algebra.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, 10 Jan 2012, Mikhail T. wrote: > I'm trying to implement a BSD-tuned sending of files. Most of the files are > fairly large. I thought, sendfile(2) is the way to go, but, it seems, there > are severe performance problems at the moment for the files located on a ZFS. > > mmap-ing the files and then write-ing them is claimed to be a lot faster. > Could somebody comment on this? Should one always mmap/write, or are there > situations, when sendfile is advantageous? Don't use mmap on zfs since doing so wastes memory (zfs ARC is not coherent with mmap page cache). Instead do normal file I/O (e.g. write, fwrite) using the filesystem blocksize (e.g. 128K) or a small multiple thereof. Of course if you are doing this over the network then you will need a program on the receiving end to write the data, and a program on the sending end to send it. It is useful to cache several blocks on the receiving end and use a thread to receive data from the network in case zfs temporarily stalls during write (which it periodically does). > If, indeed, sendfile is best for UFS, but mmap/write is better over ZFS, what > is the best way to determine the underlying FS for each file? statfs(2) is > supposed to answer that question -- what should I look for in the struct > statfs, that it will return? Do I check, if f_fsid contains a magic number > for ZFS, or look for a magic string in f_fstypename? Could someone provide an > example? Use it to obtain the filesystem block size. > Thank you very much. Yours, Good to hear from you! Bob -- Bob Friesenhahn bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?alpine.GSO.2.01.1201101741080.20764>