Date: Fri, 25 Apr 2014 09:08:41 -0400 (EDT) From: Rick Macklem <rmacklem@uoguelph.ca> To: Marek Salwerowicz <marek_sal@wp.pl> Cc: freebsd-net@freebsd.org Subject: Re: NFS over LAGG / lacp poor performance Message-ID: <31234037.791330.1398431321686.JavaMail.root@uoguelph.ca> In-Reply-To: <535A4DC2.605@wp.pl>
next in thread | previous in thread | raw e-mail | index | archive | help
Marek Salwerowicz wrote: > W dniu 2014-04-25 13:48, Rick Macklem pisze: > > Well, you don't mention what command(s) you are using to transfer > > the > > data, but I would guess you have one serial data transfer for each > > command. > > (Put another way, if you are only running one command to transfer > > the data, > > there will only be one RPC happening at a time and that will only > > use one > > network interface.) I don't know anything about lagg, so I can't > > comment > > related to it, but if there is only one NFS RPC at a time, you'll > > only > > be transferring one message at a time on the wire.) > > I need to transfer 15 files, each is about 1TB sized. > > From 9.1-RELEASE[storage1] to 10-RELEASE[storage2] > > I have tried to run concurrent 'cp' and transfer 4 files at the same > time: > > (executed on storage1) > # cp -a file1 /net/storage2/ & > # cp -a file2 /net/storage2/ & > # cp -a file3 /net/storage2/ & > # cp -a file4 /net/storage2/ & > Although I doubt it will make much difference, you might want to try "dd" with a fairly large blocksize (at least 64K). I don't know what blocksize "cp" uses and whether or not it does mmap'd file access. (mmap will only do I/O in page size blocks, so I think it will be slower.) > > But in fact I did not observe bigger throughput > > Both servers have filesystem exported using NFS, so I can execute > copy > on source, or destination. > Would you recommend running this on source-side, or rather > destination-side ? > Usually reads run faster than writes for NFS, so I'd try doing the mounts and running the commands on the destination side. That is also when "readahead=8" might help some. I'd add that option to the NFS mount (you can try any value you'd like, up to 16, but if 8 doesn't run faster than the default of 1, it probably isn't worth trying other values). > > > > Adding the mount option "readahead=8" to the machine receiving the > > data > > might help, if the data transfer command is being done there. (ie. > > The machine > > the data is being copied to has the other one NFS mounted and it is > > where > > you are running the data transfer command(s).) > > > Regarding what I wrote above - how should I mount the NFS volumes? > As above, I'd use nfsv3,readahead=8 options on the destination as a starting point. rick > Cheers, > Marek >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?31234037.791330.1398431321686.JavaMail.root>