From owner-freebsd-fs@freebsd.org Wed Aug 19 00:28:29 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F35929BC979 for ; Wed, 19 Aug 2015 00:28:28 +0000 (UTC) (envelope-from javocado@gmail.com) Received: from mail-lb0-x236.google.com (mail-lb0-x236.google.com [IPv6:2a00:1450:4010:c04::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 67562E1E for ; Wed, 19 Aug 2015 00:28:28 +0000 (UTC) (envelope-from javocado@gmail.com) Received: by lbbpu9 with SMTP id pu9so113827502lbb.3 for ; Tue, 18 Aug 2015 17:28:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=D4kCSF5dWes/xPj6WIJ5uh+rJolfDD2uXOI8SyDRKVU=; b=lIgNVORcHKSZLntfPTAlbMWf4qnhsAxv6eIKBNwD2GzGAgqqCv4TPPfe11HCiWomjJ c4T/c20ko3ioNcKb1a/gDGGyLirRjXtj89ZdYG2ayDhU0HtKs499bZtOlGlggk2Xna3F KZ0gCTflxB7BCoe3YR9Mhq1I5oxsZkmQpDX3x5842j4z70DlSYuuZpm08X+qjYBFGXQ9 49Wo1sNqz/nVT1ss3TEl4mosWCQ+HqPQWs/CFaHIbsU2oR/wOhw/FDP26pgR8IlGvN+R 8sXsvMb9gv5SxfR/uA8DpMFYc7DviL2ZpcwAEmdQfCyrRBy0Cj5TzKPAQOQefsZqJbWw kyxw== MIME-Version: 1.0 X-Received: by 10.152.43.77 with SMTP id u13mr8567106lal.96.1439944105793; Tue, 18 Aug 2015 17:28:25 -0700 (PDT) Received: by 10.114.96.136 with HTTP; Tue, 18 Aug 2015 17:28:25 -0700 (PDT) Date: Tue, 18 Aug 2015 17:28:25 -0700 Message-ID: Subject: Optimizing performance with SLOG/L2ARC From: javocado To: FreeBSD Filesystems Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2015 00:28:29 -0000 Hi, I've been trying to optimize and enhance my ZFS filesystem performance (running FreeBSD 8.3amd) which has been sluggish at times. Thus far I have added RAM (256GB) and I've added an SLOG (SSD mirror). The RAM seems to have helped a bit, but not sure if the SLOG was of much help. My vdev is decently busy, with writes and reads averaging at 100 per second with spikes as high as 500. Here's what arc_statistics is showing me: ARC Size: 70.28% 173.89 GiB Target Size: (Adaptive) 71.84% 177.77 GiB Min Size (Hard Limit): 12.50% 30.93 GiB Max Size (High Water): 8:1 247.44 GiB ARC Efficiency: 2.25b Cache Hit Ratio: 95.76% 2.16b Cache Miss Ratio: 4.24% 95.55m Actual Hit Ratio: 64.95% 1.46b Data Demand Efficiency: 94.83% 330.99m Data Prefetch Efficiency: 26.36% 64.23m CACHE HITS BY CACHE LIST: Anonymously Used: 30.87% 665.74m Most Recently Used: 7.54% 162.67m Most Frequently Used: 60.29% 1.30b Most Recently Used Ghost: 0.18% 3.97m Most Frequently Used Ghost: 1.11% 23.89m CACHE HITS BY DATA TYPE: Demand Data: 14.56% 313.89m Prefetch Data: 0.79% 16.93m Demand Metadata: 53.28% 1.15b Prefetch Metadata: 31.38% 676.68m CACHE MISSES BY DATA TYPE: Demand Data: 17.90% 17.10m Prefetch Data: 49.50% 47.30m Demand Metadata: 24.46% 23.37m Prefetch Metadata: 8.14% 7.78m 1. based on the output above, I believe a larger ARC may not necessarily benefit me at this point. True? 2. Is more (L2)ARC always better? 3. I know it's a good idea to mirror the SLOG (and I have). Do I understand correctly that I do not need to mirror the L2ARC since it's just a read cache, nothing to lose if the SSD goes down? 4. Is there a better way than looking at zpool iostat -v to determine the SLOG utilization and usefulness? I'd like to test-see if adding L2ARC yields any performance boost. Since SLOG isn't doing much for me, I'm thinking I could easily repurpose my SLOG into an L2ARC. Questions: 5. In testing, it seemed fine to remove the SLOG from a live/running system (zpool remove pool mirror-3). Is this in fact a safe thing to do to a live/running system? ZFS knows that it should flush the ZIL, then remove the device? Is it better or necessary to shut down the system and remove the SLOG in "read only" mode? 6. Am I missing something about how the SLOG and L2ARC play together that I would miss by running my proposed test. i.e. if I take down the SLOG and repurpose as an L2ARC might I be shooting myself in the foot cause the SLOG and L2ARC combo is much more powerful than the L2ARC alone (or SLOG alone)? My hope here is to see if the L2ARC improves performance, after which I will proceed with buying the SSD(s) for both the SLOG and L2ARC. Thanks