From owner-freebsd-fs@freebsd.org  Sun Jan  3 03:01:23 2016
Return-Path: <owner-freebsd-fs@freebsd.org>
Delivered-To: freebsd-fs@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5C799A5F864
 for <freebsd-fs@mailman.ysv.freebsd.org>; Sun,  3 Jan 2016 03:01:23 +0000 (UTC)
 (envelope-from mi+thun@aldan.algebra.com)
Received: from vms173005pub.verizon.net (vms173005pub.verizon.net
 [206.46.173.5])
 (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 4350415FB
 for <freebsd-fs@FreeBSD.org>; Sun,  3 Jan 2016 03:01:22 +0000 (UTC)
 (envelope-from mi+thun@aldan.algebra.com)
Received: from aldan.narawntapu ([100.1.236.52]) by vms173005.mailsrvcs.net
 (Oracle Communications Messaging Server 7.0.5.32.0 64bit (built Jul 16 2014))
 with ESMTPA id <0O0C00AWXTLFD230@vms173005.mailsrvcs.net> for
 freebsd-fs@FreeBSD.org; Sat, 02 Jan 2016 20:00:57 -0600 (CST)
X-CMAE-Score: 0
X-CMAE-Analysis: v=2.1 cv=btqxfxui c=1 sm=1 tr=0	a=UorMnhrCY2jH/mPejITChw==:117
 a=LaogzpLLAAAA:8 a=oR5dmqMzAAAA:8	a=7aQ_Q-yQQ-AA:10 a=r77TgQKjGQsHNAKrUKIA:9
 a=FnL9iEPV6B8Ad-8pvCMA:9	a=QEXdDO2ut3YA:10 a=oRl7WY1OJqzf9bcHoSsA:9
 a=_W_S_7VecoQA:10
From: "Mikhail T." <mi+thun@aldan.algebra.com>
Subject: NFS reads vs. writes
To: freebsd-fs@FreeBSD.org
Message-id: <568880D3.3010402@aldan.algebra.com>
Date: Sat, 02 Jan 2016 21:00:51 -0500
User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101
 Thunderbird/38.4.0
MIME-version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Content-Filtered-By: Mailman/MimeDel 2.1.20
X-BeenThere: freebsd-fs@freebsd.org
X-Mailman-Version: 2.1.20
Precedence: list
List-Id: Filesystems <freebsd-fs.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-fs>,
 <mailto:freebsd-fs-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-fs/>
List-Post: <mailto:freebsd-fs@freebsd.org>
List-Help: <mailto:freebsd-fs-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
 <mailto:freebsd-fs-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sun, 03 Jan 2016 03:01:23 -0000

I have two similar 10.2-STABLE machines mounting each other's
filesystems via NFS: a:/a and b:/b

When moving some large files around today, I found a huge discrepancy
between writing and reading over NFS. That is, machine a copying files
from its own /a to NFS-mounted b:/b was pushing measly 2-3Mb/s over the
Ethernet. When I cancelled that, logged-in to machine b and proceed to
copy from a:/a to the now-local /b, I got 56Mb/s. I tried changing the
wsize and rsize options, but it did not seem to make a difference...

Any ideas, what is happening here? Why are NFS-writes some 25 times
slower here, than reads? Both filesystems are zfs-backed (though with
different options), both systems are plugged into the same switch, both
use mtu of 9000.

Thanks! Yours,

    -mi