From owner-freebsd-questions@freebsd.org Thu Feb 18 21:07:33 2016 Return-Path: Delivered-To: freebsd-questions@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E9350AAD9AC for ; Thu, 18 Feb 2016 21:07:32 +0000 (UTC) (envelope-from paul@ifdnrg.com) Received: from smtp.thepharmacycentre.com (pharmacycentre.ifdnrg.com [193.200.98.77]) by mx1.freebsd.org (Postfix) with ESMTP id 9EFB71A49 for ; Thu, 18 Feb 2016 21:07:32 +0000 (UTC) (envelope-from paul@ifdnrg.com) Received: from [192.168.0.100] (cpc14-sgyl28-2-0-cust626.18-2.cable.virginm.net [82.39.86.115]) (authenticated bits=0) by smtp.thepharmacycentre.com (8.15.2/8.14.9) with ESMTPSA id u1IL7QJY072181 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Thu, 18 Feb 2016 21:07:27 GMT (envelope-from paul@ifdnrg.com) X-Authentication-Warning: smtp.thepharmacycentre.com: Host cpc14-sgyl28-2-0-cust626.18-2.cable.virginm.net [82.39.86.115] claimed to be [192.168.0.100] Subject: Re: ZFS + dovecot To: Mike Tancsa , freebsd-questions@freebsd.org References: <56C62724.60201@ifdnrg.com> <56C62F36.4050606@sentex.net> From: Paul Macdonald Message-ID: <56C632CC.7050003@ifdnrg.com> Date: Thu, 18 Feb 2016 21:08:28 +0000 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.6.0 MIME-Version: 1.0 In-Reply-To: <56C62F36.4050606@sentex.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Feb 2016 21:07:33 -0000 On 18/02/2016 20:53, Mike Tancsa wrote: > On 2/18/2016 3:18 PM, Paul Macdonald wrote: >> I'm starting to see reduced performance on a 2 disk SATA server with a >> mirrored 2TB pool running dovecot. > More memory helps with ZFS of course. How fragmented is your spool ? > Its generally a bad thing for performance once things get above 80% > capacity. Is free space greater than 20% ? > > Also, If you do > zpool get fragmentation > > How high is the fragmentation ? > 45%, probably to be expected with IMAP disk usage. (there's no way to defrag other than pool migration as i understand?) > How upto date is your OS ? 10.1-RELEASE-p26 it's kept up to date daily, > Sometimes I find with certain workloads, > setting the limit of arc is better for performance on low memory > servers. I had a box (12G of RAM) that does squid which needed > vfs.zfs.arc_max set in /boot/loader.conf so that arc would not take too > much memory. But other boxes never needed that. Not sure when it > becomes an issue and needs manual intervention. But might be something > to look at. > On our imap/pop3 server, (32G, 16G ARC limit), ARC works well > > ------------------------------------------------------------------------ > ZFS Subsystem Report Thu Feb 18 15:50:30 2016 > ------------------------------------------------------------------------ > > ARC Efficiency: 243.15m > Cache Hit Ratio: 90.12% 219.11m > Cache Miss Ratio: 9.88% 24.03m > Actual Hit Ratio: 88.46% 215.08m > > Data Demand Efficiency: 95.49% 85.08m > Data Prefetch Efficiency: 5.52% 4.81m > > CACHE HITS BY CACHE LIST: > Anonymously Used: 0.03% 59.64k > Most Recently Used: 8.51% 18.64m > Most Frequently Used: 89.66% 196.44m > Most Recently Used Ghost: 0.47% 1.03m > Most Frequently Used Ghost: 1.34% 2.94m > > CACHE HITS BY DATA TYPE: > Demand Data: 37.08% 81.25m > Prefetch Data: 0.12% 265.88k > Demand Metadata: 60.96% 133.57m > Prefetch Metadata: 1.84% 4.03m > > CACHE MISSES BY DATA TYPE: > Demand Data: 15.96% 3.83m > Prefetch Data: 18.92% 4.55m > Demand Metadata: 58.73% 14.12m > Prefetch Metadata: 6.39% 1.54m > > ------------------------------------------------------------------------ My ARC also seems to be OK, ( this after a reboot ) ------------------------------------------------------------------------ ZFS Subsystem Report Thu Feb 18 20:59:54 2016 ------------------------------------------------------------------------ ARC Efficiency: 11.11m Cache Hit Ratio: 98.90% 10.99m Cache Miss Ratio: 1.10% 122.04k Actual Hit Ratio: 98.33% 10.93m Data Demand Efficiency: 99.63% 10.23m Data Prefetch Efficiency: 12.60% 6.36k CACHE HITS BY CACHE LIST: Anonymously Used: 0.57% 63.10k Most Recently Used: 2.17% 238.03k Most Frequently Used: 97.26% 10.69m Most Recently Used Ghost: 0.00% 0 Most Frequently Used Ghost: 0.00% 0 CACHE HITS BY DATA TYPE: Demand Data: 92.72% 10.19m Prefetch Data: 0.01% 802 Demand Metadata: 6.70% 736.81k Prefetch Metadata: 0.57% 62.30k CACHE MISSES BY DATA TYPE: Demand Data: 30.64% 37.39k Prefetch Data: 4.56% 5.56k Demand Metadata: 47.03% 57.40k Prefetch Metadata: 17.77% 21.69k ------------------------------------------------------------------------ > L2 SSD is hardly touched > i was hoping for significant benefits by adding a big SSD L2ARC, your metrics suggest this could have minimal effect. > ------------------------------------------------------------------------ > ZFS Subsystem Report Thu Feb 18 15:50:57 2016 > ------------------------------------------------------------------------ > > L2 ARC Summary: (HEALTHY) > Passed Headroom: 1.94m > Tried Lock Failures: 84.50k > IO In Progress: 109 > Low Memory Aborts: 1 > Free on Write: 1.29k > Writes While Full: 611 > R/W Clashes: 0 > Bad Checksums: 0 > IO Errors: 0 > SPA Mismatch: 566.26m > > L2 ARC Size: (Adaptive) 16.23 GiB > Header Size: 0.04% 6.77 MiB > > L2 ARC Breakdown: 24.03m > Hit Ratio: 4.71% 1.13m > Miss Ratio: 95.29% 22.90m > Feeds: 592.45k > > L2 ARC Buffer: > Bytes Scanned: 41.86 TiB > Buffer Iterations: 592.45k > List Iterations: 2.37m > NULL List Iterations: 85.88k > > L2 ARC Writes: > Writes Sent: (FAULTED) 128.62k > Done Ratio: 100.00% 128.62k > Error Ratio: 0.00% 0 > > ------------------------------------------------------------------------ > > > ---Mike > are you running your services on SATA or SAS? -- ------------------------- Paul Macdonald IFDNRG Ltd Web and video hosting ------------------------- t: 0131 5548070 m: 07970339546 e: paul@ifdnrg.com w: http://www.ifdnrg.com ------------------------- IFDNRG 40 Maritime Street Edinburgh EH6 6SA ---------------------------------------------------- High Specification Dedicated Servers from £100.00pm ----------------------------------------------------