From owner-freebsd-fs@FreeBSD.ORG Fri Feb 19 07:42:39 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 82664106566B for ; Fri, 19 Feb 2010 07:42:39 +0000 (UTC) (envelope-from ndenev@gmail.com) Received: from fg-out-1718.google.com (fg-out-1718.google.com [72.14.220.154]) by mx1.freebsd.org (Postfix) with ESMTP id 11DEC8FC12 for ; Fri, 19 Feb 2010 07:42:38 +0000 (UTC) Received: by fg-out-1718.google.com with SMTP id 22so433025fge.13 for ; Thu, 18 Feb 2010 23:42:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:from:content-type :content-transfer-encoding:subject:date:message-id:to:mime-version :x-mailer; bh=VR2q7z6L7dA2n5DQe479ofbeYPDxe6/Ue2jdLwYXC7Y=; b=Ta/GF7RVBPSy0+3GBqSn70+R2HnUsQtFRzay5nvY2ZW9SVsU/XYO7E4JYhyYO4as3C xuHlAQ2XREhRUwTzHjxjOnNlWuoR+zlXoSmxa+EKjXDDHpFIJSGgiG94YXfjfsYidE86 ca82WUKCt9zpDVpz20ZaGNTzv8PPjs7l3Y0/E= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:content-type:content-transfer-encoding:subject:date:message-id :to:mime-version:x-mailer; b=ruzoXrtpQbFVYAtBMVtBXm6QeXPId4Ey2ZLuz3MG7F7iMdaYgRcZysP0qMLHXwcM9y szVQix27+I7ygJgPfncY7nga3aR6z9aRVgq8epih4RYwjgsNgJ/D6JAK7MnxF6Um3nmU IACj3OhSYPUvoofFPL7mb9TQgES+ceyZqJv04= Received: by 10.223.4.217 with SMTP id 25mr85457fas.82.1266565357910; Thu, 18 Feb 2010 23:42:37 -0800 (PST) Received: from mbp-gige.totalterror.net (93-152-151-19.ddns.onlinedirect.bg [93.152.151.19]) by mx.google.com with ESMTPS id p17sm2942670fka.5.2010.02.18.23.42.36 (version=TLSv1/SSLv3 cipher=RC4-MD5); Thu, 18 Feb 2010 23:42:36 -0800 (PST) From: Nikolay Denev Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Date: Fri, 19 Feb 2010 09:42:34 +0200 Message-Id: To: freebsd-fs@freebsd.org Mime-Version: 1.0 (Apple Message framework v1077) X-Mailer: Apple Mail (2.1077) Subject: ZFS write stalls (starving reads) and tuning zfs_write_limit_override X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Feb 2010 07:42:39 -0000 Hi, I'm experiencing a problem mentioned numerous times in the OpenSolaris = mailing lists, and that is IO stalls while ZFS commits the current TXG, which starves = all other IO. One of the solutions proposed there was to tune the = zfs_write_limit_override tunable, so the maximum TXG size could be smaller than the default (that I think = is auto-calculated from the available system memory) It seems that this is also available on FreeBSD but the tunable in = question was not declared az FreeBSD TUNABLE/SYSCTL, so I just went and added the following lines = : SYSCTL_DECL(_vfs_zfs); TUNABLE_ULONG("vfs.zfs.write_limit_override", = &zfs_write_limit_override); SYSCTL_ULONG(_vfs_zfs, OID_AUTO, zfs_write_limit_override, CTLFLAG_RW, = &zfs_write_limit_override, 0, "Override maximum TXG size"); to src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_pool.c at line = 50 With this tunable I'm now able to throttle/limit the write bursts and = this gives me a smoother iSCSI writes as opposed to the 5 sec periodic drops to zero IOPS/MBs = from before. -- Regards, Nikolay Denev