Date: Thu, 18 Feb 2016 21:08:28 +0000 From: Paul Macdonald <paul@ifdnrg.com> To: Mike Tancsa <mike@sentex.net>, freebsd-questions@freebsd.org Subject: Re: ZFS + dovecot Message-ID: <56C632CC.7050003@ifdnrg.com> In-Reply-To: <56C62F36.4050606@sentex.net> References: <56C62724.60201@ifdnrg.com> <56C62F36.4050606@sentex.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On 18/02/2016 20:53, Mike Tancsa wrote:
> On 2/18/2016 3:18 PM, Paul Macdonald wrote:
>> I'm starting to see reduced performance on a 2 disk SATA server with a
>> mirrored 2TB pool running dovecot.
> More memory helps with ZFS of course. How fragmented is your spool ?
> Its generally a bad thing for performance once things get above 80%
> capacity. Is free space greater than 20% ?
>
> Also, If you do
> zpool get fragmentation <pool name>
>
> How high is the fragmentation ?
>
45%, probably to be expected with IMAP disk usage. (there's no way to
defrag other than pool migration as i understand?)
> How upto date is your OS ?
10.1-RELEASE-p26
it's kept up to date daily,
> Sometimes I find with certain workloads,
> setting the limit of arc is better for performance on low memory
> servers. I had a box (12G of RAM) that does squid which needed
> vfs.zfs.arc_max set in /boot/loader.conf so that arc would not take too
> much memory. But other boxes never needed that. Not sure when it
> becomes an issue and needs manual intervention. But might be something
> to look at.
> On our imap/pop3 server, (32G, 16G ARC limit), ARC works well
>
> ------------------------------------------------------------------------
> ZFS Subsystem Report Thu Feb 18 15:50:30 2016
> ------------------------------------------------------------------------
>
> ARC Efficiency: 243.15m
> Cache Hit Ratio: 90.12% 219.11m
> Cache Miss Ratio: 9.88% 24.03m
> Actual Hit Ratio: 88.46% 215.08m
>
> Data Demand Efficiency: 95.49% 85.08m
> Data Prefetch Efficiency: 5.52% 4.81m
>
> CACHE HITS BY CACHE LIST:
> Anonymously Used: 0.03% 59.64k
> Most Recently Used: 8.51% 18.64m
> Most Frequently Used: 89.66% 196.44m
> Most Recently Used Ghost: 0.47% 1.03m
> Most Frequently Used Ghost: 1.34% 2.94m
>
> CACHE HITS BY DATA TYPE:
> Demand Data: 37.08% 81.25m
> Prefetch Data: 0.12% 265.88k
> Demand Metadata: 60.96% 133.57m
> Prefetch Metadata: 1.84% 4.03m
>
> CACHE MISSES BY DATA TYPE:
> Demand Data: 15.96% 3.83m
> Prefetch Data: 18.92% 4.55m
> Demand Metadata: 58.73% 14.12m
> Prefetch Metadata: 6.39% 1.54m
>
> ------------------------------------------------------------------------
My ARC also seems to be OK, ( this after a reboot )
------------------------------------------------------------------------
ZFS Subsystem Report Thu Feb 18 20:59:54 2016
------------------------------------------------------------------------
ARC Efficiency: 11.11m
Cache Hit Ratio: 98.90% 10.99m
Cache Miss Ratio: 1.10% 122.04k
Actual Hit Ratio: 98.33% 10.93m
Data Demand Efficiency: 99.63% 10.23m
Data Prefetch Efficiency: 12.60% 6.36k
CACHE HITS BY CACHE LIST:
Anonymously Used: 0.57% 63.10k
Most Recently Used: 2.17% 238.03k
Most Frequently Used: 97.26% 10.69m
Most Recently Used Ghost: 0.00% 0
Most Frequently Used Ghost: 0.00% 0
CACHE HITS BY DATA TYPE:
Demand Data: 92.72% 10.19m
Prefetch Data: 0.01% 802
Demand Metadata: 6.70% 736.81k
Prefetch Metadata: 0.57% 62.30k
CACHE MISSES BY DATA TYPE:
Demand Data: 30.64% 37.39k
Prefetch Data: 4.56% 5.56k
Demand Metadata: 47.03% 57.40k
Prefetch Metadata: 17.77% 21.69k
------------------------------------------------------------------------
> L2 SSD is hardly touched
>
i was hoping for significant benefits by adding a big SSD L2ARC, your
metrics suggest this could have minimal effect.
> ------------------------------------------------------------------------
> ZFS Subsystem Report Thu Feb 18 15:50:57 2016
> ------------------------------------------------------------------------
>
> L2 ARC Summary: (HEALTHY)
> Passed Headroom: 1.94m
> Tried Lock Failures: 84.50k
> IO In Progress: 109
> Low Memory Aborts: 1
> Free on Write: 1.29k
> Writes While Full: 611
> R/W Clashes: 0
> Bad Checksums: 0
> IO Errors: 0
> SPA Mismatch: 566.26m
>
> L2 ARC Size: (Adaptive) 16.23 GiB
> Header Size: 0.04% 6.77 MiB
>
> L2 ARC Breakdown: 24.03m
> Hit Ratio: 4.71% 1.13m
> Miss Ratio: 95.29% 22.90m
> Feeds: 592.45k
>
> L2 ARC Buffer:
> Bytes Scanned: 41.86 TiB
> Buffer Iterations: 592.45k
> List Iterations: 2.37m
> NULL List Iterations: 85.88k
>
> L2 ARC Writes:
> Writes Sent: (FAULTED) 128.62k
> Done Ratio: 100.00% 128.62k
> Error Ratio: 0.00% 0
>
> ------------------------------------------------------------------------
>
>
> ---Mike
>
are you running your services on SATA or SAS?
--
-------------------------
Paul Macdonald
IFDNRG Ltd
Web and video hosting
-------------------------
t: 0131 5548070
m: 07970339546
e: paul@ifdnrg.com
w: http://www.ifdnrg.com
-------------------------
IFDNRG
40 Maritime Street
Edinburgh
EH6 6SA
----------------------------------------------------
High Specification Dedicated Servers from £100.00pm
----------------------------------------------------
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?56C632CC.7050003>