From owner-freebsd-questions@freebsd.org Thu Feb 18 20:53:27 2016 Return-Path: Delivered-To: freebsd-questions@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AA91EAAD38D for ; Thu, 18 Feb 2016 20:53:27 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost2.sentex.ca (smarthost2.sentex.ca [IPv6:2607:f3e0:80:80::2]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "smarthost.sentex.ca", Issuer "smarthost.sentex.ca" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id B09981347 for ; Thu, 18 Feb 2016 20:53:26 +0000 (UTC) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (lava.sentex.ca [IPv6:2607:f3e0:0:5::11]) by smarthost2.sentex.ca (8.15.2/8.15.2) with ESMTPS id u1IKrPYA009741 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Thu, 18 Feb 2016 15:53:25 -0500 (EST) (envelope-from mike@sentex.net) Received: from [IPv6:2607:f3e0:0:4:5c30:ed1b:e203:c55c] ([IPv6:2607:f3e0:0:4:5c30:ed1b:e203:c55c]) by lava.sentex.ca (8.14.9/8.14.9) with ESMTP id u1IKrOm4064552; Thu, 18 Feb 2016 15:53:24 -0500 (EST) (envelope-from mike@sentex.net) Subject: Re: ZFS + dovecot To: Paul Macdonald , freebsd-questions@freebsd.org References: <56C62724.60201@ifdnrg.com> From: Mike Tancsa X-Enigmail-Draft-Status: N1110 Organization: Sentex Communications Message-ID: <56C62F36.4050606@sentex.net> Date: Thu, 18 Feb 2016 15:53:10 -0500 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.6.0 MIME-Version: 1.0 In-Reply-To: <56C62724.60201@ifdnrg.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 2.78 X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Feb 2016 20:53:27 -0000 On 2/18/2016 3:18 PM, Paul Macdonald wrote: > I'm starting to see reduced performance on a 2 disk SATA server with a > mirrored 2TB pool running dovecot. More memory helps with ZFS of course. How fragmented is your spool ? Its generally a bad thing for performance once things get above 80% capacity. Is free space greater than 20% ? Also, If you do zpool get fragmentation How high is the fragmentation ? How upto date is your OS ? Sometimes I find with certain workloads, setting the limit of arc is better for performance on low memory servers. I had a box (12G of RAM) that does squid which needed vfs.zfs.arc_max set in /boot/loader.conf so that arc would not take too much memory. But other boxes never needed that. Not sure when it becomes an issue and needs manual intervention. But might be something to look at. On our imap/pop3 server, (32G, 16G ARC limit), ARC works well ------------------------------------------------------------------------ ZFS Subsystem Report Thu Feb 18 15:50:30 2016 ------------------------------------------------------------------------ ARC Efficiency: 243.15m Cache Hit Ratio: 90.12% 219.11m Cache Miss Ratio: 9.88% 24.03m Actual Hit Ratio: 88.46% 215.08m Data Demand Efficiency: 95.49% 85.08m Data Prefetch Efficiency: 5.52% 4.81m CACHE HITS BY CACHE LIST: Anonymously Used: 0.03% 59.64k Most Recently Used: 8.51% 18.64m Most Frequently Used: 89.66% 196.44m Most Recently Used Ghost: 0.47% 1.03m Most Frequently Used Ghost: 1.34% 2.94m CACHE HITS BY DATA TYPE: Demand Data: 37.08% 81.25m Prefetch Data: 0.12% 265.88k Demand Metadata: 60.96% 133.57m Prefetch Metadata: 1.84% 4.03m CACHE MISSES BY DATA TYPE: Demand Data: 15.96% 3.83m Prefetch Data: 18.92% 4.55m Demand Metadata: 58.73% 14.12m Prefetch Metadata: 6.39% 1.54m ------------------------------------------------------------------------ L2 SSD is hardly touched ------------------------------------------------------------------------ ZFS Subsystem Report Thu Feb 18 15:50:57 2016 ------------------------------------------------------------------------ L2 ARC Summary: (HEALTHY) Passed Headroom: 1.94m Tried Lock Failures: 84.50k IO In Progress: 109 Low Memory Aborts: 1 Free on Write: 1.29k Writes While Full: 611 R/W Clashes: 0 Bad Checksums: 0 IO Errors: 0 SPA Mismatch: 566.26m L2 ARC Size: (Adaptive) 16.23 GiB Header Size: 0.04% 6.77 MiB L2 ARC Breakdown: 24.03m Hit Ratio: 4.71% 1.13m Miss Ratio: 95.29% 22.90m Feeds: 592.45k L2 ARC Buffer: Bytes Scanned: 41.86 TiB Buffer Iterations: 592.45k List Iterations: 2.37m NULL List Iterations: 85.88k L2 ARC Writes: Writes Sent: (FAULTED) 128.62k Done Ratio: 100.00% 128.62k Error Ratio: 0.00% 0 ------------------------------------------------------------------------ ---Mike -- ------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada http://www.tancsa.com/