From owner-freebsd-current@FreeBSD.ORG Sun Sep 21 17:51:26 2014 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 985E1531 for ; Sun, 21 Sep 2014 17:51:26 +0000 (UTC) Received: from mx1.scaleengine.net (beauharnois2.bhs1.scaleengine.net [142.4.218.15]) by mx1.freebsd.org (Postfix) with ESMTP id 558973A5 for ; Sun, 21 Sep 2014 17:51:25 +0000 (UTC) Received: from [192.168.1.2] (Seawolf.HML3.ScaleEngine.net [209.51.186.28]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by mx1.scaleengine.net (Postfix) with ESMTPSA id 714FB4F117 for ; Sun, 21 Sep 2014 17:51:24 +0000 (UTC) Message-ID: <541F101A.6000701@freebsd.org> Date: Sun, 21 Sep 2014 13:51:22 -0400 From: Allan Jude User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.1.1 MIME-Version: 1.0 To: freebsd-current@freebsd.org Subject: Re: zpool frag References: <1411289830171-5950788.post@n5.nabble.com> <1691600.4gjp5IhhyR@overcee.wemm.org> <1965644.89SVIqCBMH@overcee.wemm.org> In-Reply-To: <1965644.89SVIqCBMH@overcee.wemm.org> Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="xOFUeXpGFAUIqdh5w5pkivvHfEECAw5VO" X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Sep 2014 17:51:26 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --xOFUeXpGFAUIqdh5w5pkivvHfEECAw5VO Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 2014-09-21 13:39, Peter Wemm wrote: > On Sunday, September 21, 2014 06:12:09 PM Steven Hartland wrote: >> ----- Original Message ----- >> >>> From: "Peter Wemm" >>> >>> On Sunday, September 21, 2014 11:06:10 AM Allan Jude wrote: >>>> On 2014-09-21 04:57, Beeblebrox wrote: >>>>> FRAG means fragmentation, right? Zpool fragmentation? That's news t= o >>>>> me. >>>>> If >>>>> this is real how do I fix it? >>>>> >>>>> NAME SIZE ALLOC FREE FRAG EXPANDSZ CAP DEDUP HEALTH= >>>>> ALTROOT pool1 75.5G 53.7G 21.8G 60% - 71% 1.0= 0x >>>>> ONLINE - pool2 48.8G 26.2G 22.6G 68% - 53%=20 >>>>> 1.00x >>>>> ONLINE - pool3 204G 177G 27.0G 53% - 86%=20 >>>>> 1.11x >>>>> ONLINE - >>>> >>>> It is not something you 'fix', it is just a metric to help you >>>> understand the performance of your pool. The higher the fragmentatio= n, >>>> the longer it might take to allocate new space, and obviously you wi= ll >>>> have more random seek time while reading from the pool. >>>> >>>> As Steven mentions, there is no defragmentation tool for ZFS. You ca= n >>>> zfs send/recv or backup/restore the pool if you have a strong enough= >>>> reason to want to get the fragmentation number down. >>>> >>>> It is a fairly natural side effect of a copy-on-write file system. >>>> >>>> Note: the % is not the % fragmented, IIRC, it is the percentage of t= he >>>> free blocks that are less that a specific size. I forget what that s= ize >>>> is. >>> >>> I fear that the information presented in its current form is going to= >>> generate lots of fear and confusion. >>> >>> The other thing to consider is that this gets much, much worse as the= pool >>> fills up. Even UFS has issues with fragmentation when it fills, but = ZFS >>> is far more sensative to it. In the freebsd.org cluster we have a he= alth >>> check alert at 80% full, but even that's probably on the high side. >> >> This "should" be less of an issue if you have the spacemap_histogram f= eature >> enabled on the pool, which IIRC if your seeing FRAG details should be = the >> case. >=20 > Hopefully so. The catch though is when its been run without it until r= ecently=20 > it can be a bit of a surprise. >=20 >=20 =46rom an email exchange with George Wilson (developer of the spacemap_histogram feature): "You can use it on existing pools but the histogram only gets created when you condense a space_map (a process by which ZFS tries to make the space_map ondisk smaller). This means that when you look at an existing pool it may have some space_maps with histograms and ones without." This likely means that on a pool that has (recently??) been upgrade, the FRAG metric may fluctuate wildly, as more and more space_maps get a histogram. The histogram feature doesn't really solve the problem of fragmentation, it just makes the pool perform better when it is fragmented, as the pool can more reliably find the space_map containing the largest area of contiguous free space in order to maintain performance. --=20 Allan Jude --xOFUeXpGFAUIqdh5w5pkivvHfEECAw5VO Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQIcBAEBAgAGBQJUHxAeAAoJEJrBFpNRJZKf+sYP/3HoqWRfbj86sB6zIWJYgnAO 4U3hEMQlMljSTMptTwv+mR7Rj2wBxL/mIKyeNJD0agrJ+1FsHlahYcBwqrzPCq6S oGAlIPYLpBW3TL/lpS1XC6SyR34JJU4EFHDHqvMsXq47YLUTa9wd2IEsayvCRt9/ m2AteSpC16UEoALUXRRO97qThzbBmmtA0faMjQskL3wsNHfjb4MCG0YR0esge4cI ZdtxYM/1/GHYICzmFz67zjAYE8hrW/BAssXk07gEHD78zZe8Hvm6fsy5rP54o0Wa eC4OM0+wCStqq3yzLjvXIH5AailJOj5CtsDmCJBhqoBNGdVGfSvLmb/BrwKmFzyE IZ2tdOMa8R5Ymb8rul+7By3+3LoZgPnHIm2nPBjtWK75IeKsKTlE31xx4OYdF/Zn hWDAP5ERiLTn/Dq2dGNEDBRPUnNYPa9JbuhZ2LyWw44qOe2boeBi8wruy8T1tkbF lZWCbKbWwUZLrXpDYcqwHz+6iHVhLfgHhcm1x9fAfBaLTbCsxHqN85xEaCHZ047w imRnl1nr2AZD01G5tbuU7tnupen3usSx68IeSEfQSEdxbnO7vhAFPL9sBzHx31yq h5LFptYXqQcJ/GryhZg0C13TMadyXY4aMcWUUDDn9ZPE028Vgqv6s/JFWtrr8xgk 0SfSSJhRQtwjttt2EhZ+ =6SBK -----END PGP SIGNATURE----- --xOFUeXpGFAUIqdh5w5pkivvHfEECAw5VO--