Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 18 Feb 2016 21:08:28 +0000
From:      Paul Macdonald <paul@ifdnrg.com>
To:        Mike Tancsa <mike@sentex.net>, freebsd-questions@freebsd.org
Subject:   Re: ZFS + dovecot
Message-ID:  <56C632CC.7050003@ifdnrg.com>
In-Reply-To: <56C62F36.4050606@sentex.net>
References:  <56C62724.60201@ifdnrg.com> <56C62F36.4050606@sentex.net>

Next in thread | Previous in thread | Raw E-Mail | Index | Archive | Help

On 18/02/2016 20:53, Mike Tancsa wrote:
> On 2/18/2016 3:18 PM, Paul Macdonald wrote:
>> I'm starting to see reduced performance on a 2 disk SATA server with a
>> mirrored 2TB pool running dovecot.
> More memory helps with ZFS of course.  How fragmented is your spool ?
> Its generally a bad thing for performance once things get above 80%
> capacity.  Is free space greater than 20% ?
>
> Also,  If you do
> zpool get fragmentation <pool name>
>
> How high is the fragmentation ?
>

45%, probably to be expected with IMAP disk usage.   (there's no way to 
defrag other than pool migration as i understand?)

> How upto date is your OS ?

10.1-RELEASE-p26
it's kept up to date daily,

> Sometimes I find with certain workloads,
> setting the limit of arc is better for performance on low memory
> servers. I had a box (12G of RAM) that does squid which needed
> vfs.zfs.arc_max set in /boot/loader.conf so that arc would not take too
> much memory.  But other boxes never needed that. Not sure when it
> becomes an issue and needs manual intervention.  But might be something
> to look at.

> On our imap/pop3 server,  (32G, 16G ARC limit), ARC works well
>
> ------------------------------------------------------------------------
> ZFS Subsystem Report                            Thu Feb 18 15:50:30 2016
> ------------------------------------------------------------------------
>
> ARC Efficiency:                                 243.15m
>          Cache Hit Ratio:                90.12%  219.11m
>          Cache Miss Ratio:               9.88%   24.03m
>          Actual Hit Ratio:               88.46%  215.08m
>
>          Data Demand Efficiency:         95.49%  85.08m
>          Data Prefetch Efficiency:       5.52%   4.81m
>
>          CACHE HITS BY CACHE LIST:
>            Anonymously Used:             0.03%   59.64k
>            Most Recently Used:           8.51%   18.64m
>            Most Frequently Used:         89.66%  196.44m
>            Most Recently Used Ghost:     0.47%   1.03m
>            Most Frequently Used Ghost:   1.34%   2.94m
>
>          CACHE HITS BY DATA TYPE:
>            Demand Data:                  37.08%  81.25m
>            Prefetch Data:                0.12%   265.88k
>            Demand Metadata:              60.96%  133.57m
>            Prefetch Metadata:            1.84%   4.03m
>
>          CACHE MISSES BY DATA TYPE:
>            Demand Data:                  15.96%  3.83m
>            Prefetch Data:                18.92%  4.55m
>            Demand Metadata:              58.73%  14.12m
>            Prefetch Metadata:            6.39%   1.54m
>
> ------------------------------------------------------------------------

My ARC also seems to be OK, ( this after a reboot )

------------------------------------------------------------------------
ZFS Subsystem Report                            Thu Feb 18 20:59:54 2016
------------------------------------------------------------------------

ARC Efficiency:                                 11.11m
         Cache Hit Ratio:                98.90%  10.99m
         Cache Miss Ratio:               1.10%   122.04k
         Actual Hit Ratio:               98.33%  10.93m

         Data Demand Efficiency:         99.63%  10.23m
         Data Prefetch Efficiency:       12.60%  6.36k

         CACHE HITS BY CACHE LIST:
           Anonymously Used:             0.57%   63.10k
           Most Recently Used:           2.17%   238.03k
           Most Frequently Used:         97.26%  10.69m
           Most Recently Used Ghost:     0.00%   0
           Most Frequently Used Ghost:   0.00%   0

         CACHE HITS BY DATA TYPE:
           Demand Data:                  92.72%  10.19m
           Prefetch Data:                0.01%   802
           Demand Metadata:              6.70%   736.81k
           Prefetch Metadata:            0.57%   62.30k

         CACHE MISSES BY DATA TYPE:
           Demand Data:                  30.64%  37.39k
           Prefetch Data:                4.56%   5.56k
           Demand Metadata:              47.03%  57.40k
           Prefetch Metadata:            17.77%  21.69k

------------------------------------------------------------------------


> L2 SSD is hardly touched
>
i was hoping for significant benefits by adding a big SSD L2ARC, your 
metrics suggest this could have minimal effect.

> ------------------------------------------------------------------------
> ZFS Subsystem Report                            Thu Feb 18 15:50:57 2016
> ------------------------------------------------------------------------
>
> L2 ARC Summary: (HEALTHY)
>          Passed Headroom:                        1.94m
>          Tried Lock Failures:                    84.50k
>          IO In Progress:                         109
>          Low Memory Aborts:                      1
>          Free on Write:                          1.29k
>          Writes While Full:                      611
>          R/W Clashes:                            0
>          Bad Checksums:                          0
>          IO Errors:                              0
>          SPA Mismatch:                           566.26m
>
> L2 ARC Size: (Adaptive)                         16.23   GiB
>          Header Size:                    0.04%   6.77    MiB
>
> L2 ARC Breakdown:                               24.03m
>          Hit Ratio:                      4.71%   1.13m
>          Miss Ratio:                     95.29%  22.90m
>          Feeds:                                  592.45k
>
> L2 ARC Buffer:
>          Bytes Scanned:                          41.86   TiB
>          Buffer Iterations:                      592.45k
>          List Iterations:                        2.37m
>          NULL List Iterations:                   85.88k
>
> L2 ARC Writes:
>          Writes Sent: (FAULTED)                          128.62k
>            Done Ratio:                   100.00% 128.62k
>            Error Ratio:                  0.00%   0
>
> ------------------------------------------------------------------------
>
>
> 	---Mike
>

are you running your services on SATA or SAS?

-- 
-------------------------
Paul Macdonald
IFDNRG Ltd
Web and video hosting
-------------------------
t: 0131 5548070
m: 07970339546
e: paul@ifdnrg.com
w: http://www.ifdnrg.com
-------------------------
IFDNRG
40 Maritime Street
Edinburgh
EH6 6SA
----------------------------------------------------
High Specification Dedicated Servers from £100.00pm
----------------------------------------------------




Want to link to this message? Use this URL: <http://docs.FreeBSD.org/cgi/mid.cgi?56C632CC.7050003>