From owner-freebsd-fs@FreeBSD.ORG Wed Mar 27 10:14:59 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 9A33C7B7 for ; Wed, 27 Mar 2013 10:14:59 +0000 (UTC) (envelope-from tevans.uk@googlemail.com) Received: from mail-la0-x22a.google.com (mail-la0-x22a.google.com [IPv6:2a00:1450:4010:c03::22a]) by mx1.freebsd.org (Postfix) with ESMTP id 2B68BB60 for ; Wed, 27 Mar 2013 10:14:58 +0000 (UTC) Received: by mail-la0-f42.google.com with SMTP id fe20so15363644lab.1 for ; Wed, 27 Mar 2013 03:14:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=U4qFRvKLj8SnMwpZ6DS5sXZrMKtR53ICFKmmkFYSkww=; b=rLrX9fMoVmAtcrNjsX6Md0ixUa0GXAFWcoRxQ8C4qnRBr9lHfDUVortKlyCPwyaEXg OBBvUg1mACAymr8fopbXh4JtybDyJAg0UXBQI2UU6oXwvndVsGYJU3BH1KHEItCL5Fmk wXbVrpDssTNWCE5bEtxFNnRG27IMplxc+pPsYx61Ub7UuxUgn6ExdnI7hk1iiZLd29Dq h+2/7NhkgaDmJzxQ3iCCphtSOKs04aL6Z/M0F+iyZ61O1WvNnrX9q4lBInWvKlvTYeBO Qz7ZOp+losI+BJcZuDfBYYxcKHVkLga0fqyLnECY69q+uGC0qE1bGRW9ZMLsbVOEmTp5 tyeA== MIME-Version: 1.0 X-Received: by 10.152.133.52 with SMTP id oz20mr10057682lab.30.1364379298105; Wed, 27 Mar 2013 03:14:58 -0700 (PDT) Received: by 10.112.26.135 with HTTP; Wed, 27 Mar 2013 03:14:57 -0700 (PDT) In-Reply-To: <20130322172657.95912@relay.ibs.dn.ua> References: <20130322172657.95912@relay.ibs.dn.ua> Date: Wed, 27 Mar 2013 10:14:57 +0000 Message-ID: Subject: Re: RAM amount recommendations for ZFS pools with ZIL and L2ARC on SSD From: Tom Evans To: Zeus Panchenko Content-Type: text/plain; charset=UTF-8 Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Mar 2013 10:14:59 -0000 On Fri, Mar 22, 2013 at 2:26 PM, Zeus Panchenko wrote: > hi all, > > while discovering the subj, I found recommendations at: > http://doc.freenas.org/index.php/Hardware_Recommendations#RAM > > - --------------------------------------------------------------------- > ... a general rule of thumb is 1 GB of RAM for every 1TB of storage ... > If you plan to use ZFS deduplication, a general rule of thumb is 5 GB > RAM per TB of storage to be deduplicated. > - --------------------------------------------------------------------- > > > so, are these recommendations correct for ZFS pools with ZIL and L2ARC > on SSD configurations? > > are there corellations between "RAM amount" and "with/without ZIL, L2ARC > on separate devices pool configuration" ? > Well, they wouldn't hurt. Personally, I've used much less than that - I have a 12 x 1.5 TB server with 8 GB RAM, which is only 0.5GB/TB. It's replacement will have significantly more, but only because RAM is cheap(er). Dedupe is a special beast, to get decent performance the whole dedupe table must fit into memory, Oracle have a sizing guide: http://www.oracle.com/technetwork/articles/servers-storage-admin/o11-113-size-zfs-dedup-1354231.html If you don't have a ZIL, a portion of your pool is used instead as ZIL. RAM is never used for ZIL. Remember that the ZIL is only for synchronous writes, which most writes are not. RAM is mainly used for the ARC cache, and RAM is a lot faster than an SSD, so the more RAM used for the ARC cache the better. Adding an SSD L2ARC in addition will increase the amount of things that can be kept cached in a (relatively) slow cache. If your problem is that you need that data in a faster cache, then you need more RAM. I think I am right in saying that only things that have been read can be stored in ARC, so how effective it is would depend upon your workload - how much read things do you need in the cache? Cheers Tom