Date: Fri, 25 Sep 2015 21:24:45 +0100 From: Matthew Seaman <matthew@FreeBSD.org> To: freebsd-questions@freebsd.org Subject: Re: Problems with ZFS file servers Message-ID: <5605AD8D.4080706@FreeBSD.org> In-Reply-To: <5605A3A5.7080905@rpi.edu> References: <55F5CF06.5080602@rpi.edu> <1442191342.426007007.amnhjwng@frv35.fwdcdn.com> <55F62523.7010402@rpi.edu> <1442267713.360040098.um8wwy4s@frv35.fwdcdn.com> <5605A3A5.7080905@rpi.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --bMkHs9PXFiKXUg6QQGveIkcXxM4M2LAjO Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 25/09/2015 20:42, Bob Healey wrote: > I've got another machine acting up, this machine is a Sun X2250, > originally was running Solaris until this summer when the owner dropped= > the support contract. A zpool export, reinstall to FreeBSD 10.1, and > zpool import and it was back in business. Most of the requested info > can be found at http://origami.phys.rpi.edu/~healer/lepton. I am > working on getting cacti installed. I am currently trying to rsync a > workstation to this pool so I can reload the OS. If I use > --bwlimit=3D10240 or lower, I have no issues, but if I don't rsync free= zes > on me on the client side. I've found, through bitter experience, that you need to apply some tunings to ZFS machines, and quite possibly some kernel patches too. When you're pumping wads of data into a ZFS machine at high speed, it is all too easy to get it to lock up. First up, the default setting where ZFS grabs all but 1GB of available RAM for use by the ARC is nuts. You need to chop that down and give the rest of the OS a fair share of RAM to play with by setting vfs.zfs.arc_max in /boot/loader.conf. What you set it to depends on the application mix on your server, but somewhere around 50% of available RAM seems reasonable to me. Reboot to enable that, obviously. If you're dumping a lot of data (especially if you're writing much more than will fit into system RAM) onto your ZFS box, and don't intend to be using that data actively any time soon, then it's a good idea to set properties to disable the primarycache and secondarycache on the ZFS you're writing to. Or set them to metadata only. Otherwise you'll just blow out your ARC caches which won't help system performance. You can turn caching on or off for specific ZFSes at any time. If that doesn't do enough to make your machine work reliably, then investigate using the patches from PR 187594: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D187594 There are several versions of the patch but you should be able to find one that applies to the version of the OS you're using. Cheers, Matthew --bMkHs9PXFiKXUg6QQGveIkcXxM4M2LAjO Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2 iQJ8BAEBCgBmBQJWBa2UXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQ2NTNBNjhCOTEzQTRFNkNGM0UxRTEzMjZC QjIzQUY1MThFMUE0MDEzAAoJELsjr1GOGkAT5RIP/RqqZ/kpALiAqRxP+6/G01P3 D+necGn0gfYnEoXSfCjxxCBlcl2X159vKP+vJ5F27cOVTzUJUAhkhhP228G1X71A zOPcUCNQEcYnN+AJ3InNLtm8cnBarBJBBIWScJyZl3jMlOoIy62Q3EkHeCo7X9P2 AW61gulq4KpuloioOjRbT94nYNWY/WXlZWZOXh/lZvUHvSnoZmoL9VxYD+VpTKSz Jg0EQfA4uyzkGoeyvpOWlNrlsxD5yFqgpxSxL5uRcq6mbeNWnRuQ/FsdXav3SME2 pCC7EYDrjiXuLl7Bd/jLnpmFaIUThTxYV+Nn+klCuBav/+4vWWtlsNcj1Pju5Akl LFoy69YEzhSxPSRGA8IuYRMZqgaeBduNlN7v6p4yqBVy7+Xt/CP2EQjR5uBRcT9C TdLJVQXi1iz0LLrCJWPZSbeCEhR5gDEfYxs0cdJNPYNy4UyetSCvoRJSANQhZ8YD r1nsh5r752bwJH7K/X0Nwz2/HwXBdZpCsYgM/V/lMMYm89IcFen9NTaG/GMZIn9U wkGJXS+IBBQNyZ3ZEa2VDt/m6LknFEzqn0fK2LQwZBI+AJiAHZCyZjE1CJINgA8W FNjM1iJO/Ej4rNc05ZkJs4OMHg2c1rPrJSbc4HXzpE9UPwEWILS92ab3yf6YsVVx U4YAdC5XEHPWZWwdI2NH =oGQA -----END PGP SIGNATURE----- --bMkHs9PXFiKXUg6QQGveIkcXxM4M2LAjO--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5605AD8D.4080706>