From owner-freebsd-fs@FreeBSD.ORG  Wed Jan 16 07:42:46 2013
Return-Path: <owner-freebsd-fs@FreeBSD.ORG>
Delivered-To: freebsd-fs@freebsd.org
Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115])
 by hub.freebsd.org (Postfix) with ESMTP id 92A1CC74;
 Wed, 16 Jan 2013 07:42:46 +0000 (UTC)
 (envelope-from nicolas@i.0x5.de)
Received: from n.0x5.de (n.0x5.de [217.197.85.144])
 by mx1.freebsd.org (Postfix) with ESMTP id 179DDF98;
 Wed, 16 Jan 2013 07:42:46 +0000 (UTC)
Received: by pc5.i.0x5.de (Postfix, from userid 1003)
 id 3YmL452m1Yz7ySF; Wed, 16 Jan 2013 08:42:45 +0100 (CET)
Date: Wed, 16 Jan 2013 08:42:45 +0100
From: Nicolas Rachinsky <fbsd-mas-0@ml.turing-complete.org>
To: Steven Hartland <killing@multiplay.co.uk>
Subject: Re: slowdown of zfs (tx->tx)
Message-ID: <20130116074245.GB47781@mid.pc5.i.0x5.de>
References: <CAFqOu6gWpMsWN0pTBiv10WfwyGWMfO9GzMLWTtcVxHixr-_i3Q@mail.gmail.com>
 <20130114094010.GA75529@mid.pc5.i.0x5.de>
 <CAFqOu6hxfGt_M6Jo9qWeifDz9YnNc_Bd9H-GEe4RYtutaPvH5w@mail.gmail.com>
 <20130114195148.GA20540@mid.pc5.i.0x5.de>
 <CAFqOu6jwJ4qhbOovN_NhzusdQJvrbvUC3g93sziR=Uw99SGenw@mail.gmail.com>
 <20130114214652.GA76779@mid.pc5.i.0x5.de>
 <CAFqOu6jKX-Ks6C1RK5GwZ51ZVUSnGSe7S99_EfK+fwLPjAFFYw@mail.gmail.com>
 <20130115224556.GA41774@mid.pc5.i.0x5.de>
 <CAFqOu6jJnWdbikPmE1-UML5i_x7meF+iyY=9WBRyv2j7AeOaSg@mail.gmail.com>
 <00F86FD0E85D4EEEA1A01E115497F022@multiplay.co.uk>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <00F86FD0E85D4EEEA1A01E115497F022@multiplay.co.uk>
X-Powered-by: FreeBSD
X-Homepage: http://www.rachinsky.de
X-PGP-Keyid: 887BAE72
X-PGP-Fingerprint: 039E 9433 115F BC5F F88D  4524 5092 45C4 887B AE72
X-PGP-Keys: http://www.rachinsky.de/nicolas/gpg/nicolas_rachinsky.asc
User-Agent: Mutt/1.5.21 (2010-09-15)
Cc: freebsd-fs <freebsd-fs@freebsd.org>
X-BeenThere: freebsd-fs@freebsd.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: Filesystems <freebsd-fs.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-fs>,
 <mailto:freebsd-fs-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-fs>
List-Post: <mailto:freebsd-fs@freebsd.org>
List-Help: <mailto:freebsd-fs-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
 <mailto:freebsd-fs-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 16 Jan 2013 07:42:46 -0000

* Steven Hartland <killing@multiplay.co.uk> [2013-01-16 00:57 -0000]:
> 
> ----- Original Message ----- From: "Artem Belevich"
> <art@freebsd.org>
> To: "Nicolas Rachinsky" <fbsd-mas-0@ml.turing-complete.org>
> Cc: "freebsd-fs" <freebsd-fs@freebsd.org>
> Sent: Wednesday, January 16, 2013 12:16 AM
> Subject: Re: slowdown of zfs (tx->tx)
> 
> 
> >It appears that lots of threads are stuck in
> >metaslab_activate->space_map_load_wait path. This sounds like CR#
> >6876962 in Solaris: "degraded write performance with threads held up
> >by space_map_load_wait(). This bug is fixed in patch 147440-05, -06 or
> >-07, which is current and contains the fix." Alas, I could not find
> >specifics on how the issue got fixed and whether the same fix is
> >present in illumos and FreeBSD.
> 
> That would tend to indicate its blocking on write. If this is the case
> yet the rsync is copying from this box, with little else doing writes
> it could be atime which is causing the issue.

I was probably misformulating my mail. The rsync writes to the local
zpool.

> A test for this would be to use the following to disable atime and see
> if that helps:
> zfs set atime=off [filesystem]
> 
> Also out of interest does the pool have many snapshots?

There are 115 filesystems. 84 of these have between 10 and 20
snapshots.

Nicolas
-- 
http://www.rachinsky.de/nicolas