Date: Mon, 13 Jun 2011 15:35:29 -0400 From: jhell <jhell@DataIX.net> To: Steven Hartland <killing@multiplay.co.uk> Cc: freebsd-fs@freebsd.org Subject: Re: Impossible compression ratio on ZFS Message-ID: <20110613193529.GA21103@DataIX.net> In-Reply-To: <4E09C82B45BA46019281930B2EB13AC1@multiplay.co.uk> References: <F21D6DCDBA494B4A9FDF20A13BC4947A@multiplay.co.uk> <20110613094803.GA10290@icarus.home.lan> <4E09C82B45BA46019281930B2EB13AC1@multiplay.co.uk>
next in thread | previous in thread | raw e-mail | index | archive | help
--sdtB3X0nJg68CQEu Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jun 13, 2011 at 11:57:22AM +0100, Steven Hartland wrote: > ----- Original Message -----=20 > From: "Jeremy Chadwick" <freebsd@jdc.parodius.com> > =20 > > Well-known "quirk"; welcome to ZFS. :-) The following article is long, > > but if you grab a coffee and read it in full, it'll shed some light on > > the ordeal: > >=20 > > http://www.cuddletech.com/blog/pivot/entry.php?id=3D983 > >=20 > > There's also this: > >=20 > > http://blog.buttermountain.co.uk/2008/05/10/zfs-compression-when-du-and= -ls-appear-to-disagree/ > >=20 > > This is one of the many reasons I do not use ZFS compression. Not > > spreading FUD, just saying stuff like this throws users for a loop, case > > in point. >=20 > I think your miss-understanding my question, its not the fact that its > showing different sizes from du and ls, that's 100% expected but clearly > 8million rows of 3 int's can't possibly compress down to 7.5K. >=20 > Having just looked back at the machine, an hour later, the values now > seem correct with du showing:- > 278M detail.ibd >=20 > I checked this several times, over what had to be 10mins or more even > did a flush tables to ensure everything had been written out as far > as mysql was concerned. >=20 > So it seems that zfs was still processing the file for a good amount of > time, and during that time was showing incorrect disk usage for said file. >=20 > I'm wondering if the data is some how being processed in l2 arc or > something? >=20 > For reference we're running 8.2-RELEASE, on an areca backed raid6 with > two ssd drives in l2 arc. >=20 > zpool status > pool: tank > state: ONLINE > scrub: none requested > config: >=20 > NAME STATE READ WRITE CKSUM > tank ONLINE 0 0 0 > da0p3 ONLINE 0 0 0 > cache > ada0 ONLINE 0 0 0 > ada1 ONLINE 0 0 0 >=20 > errors: No known data errors >=20 > Obviously everything seems to have caught up and is now showing real > stats but confused as to why it would take quite so long to display > the real usage via du. >=20 Hi Steve, Knowing that there were patches out for v28 on 8.X can you confirm that in fact you are using v15 ZFS ? I would assume you are because of the release but I don't want to do that. If you happen to have patched up to v28 did you turn dedup on.? if so I would expect the behavior your seeing with the data not being written right away. If not, then seeing you have compression turned on... did you just dump that whole table into the database ? its quite possible that the compression was still happening in ARC before it was finally written out and this would also explain why that happened. Also what level of compression are you using ? --sdtB3X0nJg68CQEu Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (FreeBSD) Comment: http://bit.ly/0x89D8547E iQEcBAEBAgAGBQJN9mZ/AAoJEJBXh4mJ2FR+6ucH/RW82Bh9i0AAJ56m3Ojx+GqY BdoizrBBoJrxAqu+XpvMU/P4B94TAfe921ZOE1GH9fy2eZzthh9uzQ/329+BqJ5Z Lnvq0AdmZFfO2xFiGvABnBkBNSCXQUNM/Yh4EGpKXZmP5Ga69o8845Fm++0xC4sr 7pyrXNUTpQtDUfN/BABnjE52MA6VUxVUPsSqMnQ/ugN5fLLOmHbKJETCoPkBRdEX +aZEBAmikP02Y+K+Jo5YseWy92m/B2pH2DTVMZN9nyoZVbgppeacEasG09Kl4q02 gnNDsGd4lNBgFtg3akev2so7xDmB2FzX5dBGSuOSMhfJ7uGhcBeNHz5k+geUJ34= =a5WH -----END PGP SIGNATURE----- --sdtB3X0nJg68CQEu--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20110613193529.GA21103>