From owner-freebsd-performance@FreeBSD.ORG Mon Apr 12 22:15:30 2010 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 45F85106564A; Mon, 12 Apr 2010 22:15:30 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from mail.digiware.nl (mail.ip6.digiware.nl [IPv6:2001:4cb8:1:106::2]) by mx1.freebsd.org (Postfix) with ESMTP id D1FD58FC08; Mon, 12 Apr 2010 22:15:29 +0000 (UTC) Received: from localhost (localhost.digiware.nl [127.0.0.1]) by mail.digiware.nl (Postfix) with ESMTP id 9C6BE15346D; Tue, 13 Apr 2010 00:15:27 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.nl Received: from mail.digiware.nl ([127.0.0.1]) by localhost (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1veTU93reVMQ; Tue, 13 Apr 2010 00:15:25 +0200 (CEST) Received: from [127.0.0.1] (unknown [192.168.10.212]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.digiware.nl (Postfix) with ESMTPSA id 0109015346C; Tue, 13 Apr 2010 00:15:24 +0200 (CEST) Message-ID: <4BC39B8C.6020808@digiware.nl> Date: Tue, 13 Apr 2010 00:15:40 +0200 From: Willem Jan Withagen User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.9) Gecko/20100317 Thunderbird/3.0.4 MIME-Version: 1.0 To: kmacy@freebsd.org References: <20100408153257.GA35557@runo.ripn.net> <20100409024409.GA1602@runo.ripn.net> <20100412091136.GE61157@runo.ripn.net> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Mon, 12 Apr 2010 22:23:26 +0000 Cc: freebsd-performance@freebsd.org, "Alexandr D. Sergeev" Subject: Re: Extreme performance falling on ZFS by durable operation of a server X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Apr 2010 22:15:30 -0000 On 12-4-2010 21:56, K. Macy wrote: > 8-STABLE is definitely better, but you should also set arc_min to at > least 384M-512M. I'm not sure that arc_min is even honored under 7.2. Your mileage might vary.... I've been tempted to report similar problems running 8.0-stable march 23rd on amd64 with 8Gb running as fileserver (nfs/smb), imap/dovecot, rsync backup But I get stalls (of several seconds) when trying to do something in the shell first thing in the morning. After running some more service responce gets a bit more back to normal. Same holds for access over SMB and NFS with suffers from delays as well. Seen more of these type of reports with unconclusive answers thusfar, several of the suggestions did not improve the situation. Haven't found the time to very seriously trace this, and report more than just this. --WjW > On Mon, Apr 12, 2010 at 2:11 AM, Alexandr D. Sergeev wrote: >> Whether there are any ideas about the reasons of such problems? >> >> Whether transition to FreeBSD 8.0 RELEASE / STABLE can help? >> ZFS in it is declared as "bug free" >> >> On Fri, Apr 09, 2010 at 06:44:09AM +0400, Alexandr D. Sergeev wrote: >>> On Thu, Apr 08, 2010 at 01:14:42PM -0700, K. Macy wrote: >>>> Hi Alexandr, >>> >>> Hi Kip, >>> >>>> >>>> - How much memory do you have? >>> >>> 4Gb. >>> Applications consume a lot of memory, is partially used swap (~1Gb, but without active i/o). >>> >>> We have planned increase in memory to 16G >>> >>>> >>>> - What are your ARC settings? >>> >>> Settings is default, that recommended for amd64 kernel. >>> But I am not assured that it is optimal. >>> Thus sysctl variables are automatically sets to: >>> vfs.zfs.arc_max ~800Mb >>> vfs.zfs.arc_mim ~128Mb >>> >>> Now the next reboot was required. >>> Together with which following settings have been made: >>> >>> vm.kmem_size=2048M >>> vfs.zfs.arc_max=1536M >>> vfs.zfs.arc_min=1024M >>> vfs.zfs.vdev.cache.size=64M >>> >>> Let's look, as it will affect. >>> >>> However there is confidence that no problem at a size of ARC. >>> Even at 200Mb ARC, productivity should not fall so strongly. >>> >>>> >>>> Thanks, >>>> Kip >>>> >>>> On Thu, Apr 8, 2010 at 8:32 AM, Alexandr D. Sergeev wrote: >>>>> Hi, all! >>>>> >>>>> There are problems with a server on FreeBSD 7.2 CURRENT (6 Jul 2009), carrying out the task of mail server SMTP, IMAP. >>>>> >>>>> Messages are stored in maildirs on file system ZFS (mirror of 2 SAS 3k drives). >>>>> Dovecot is used as IMAP server. >>>>> >>>>> After reboot the system works perfectly, however, somewhere in a month, at the same load, arise periodic long delays at opening of new letters. >>>>> >>>>> Judging by output iostat -x -w 1, during these moments the disk subsystem is strongly loaded. >>>>> %busy disks comes nearer to 100 %. Readings nearby 300 per sec prevail. and approx. 10MBytes per sec. >>>>> >>>>> Somebody can advise, what diagnostics needed, for understanding of the reason of problems? >>>>> >>>>> There is an opinion that a problem with ZFS. >>>>> >>>>> Probably, it does not have enough memory. Because during the periods of occurrence of a problem, ARC is have less 300MB: >>>>> >>>>> vmstat -m >>>>> ... >>>>> solaris 1245732 274745K - 25872717949 16,32,64,128,256,512,1024,2048,4096 >>>>> ... >>>>> >>>>> Or it is bug of ZFS implementation. >>>>> >>>>> -- >>>>> Best regards, >>>>> Alexandr D.Sergeev >>>>> _______________________________________________ >>>>> freebsd-performance@freebsd.org mailing list >>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-performance >>>>> To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd.org" >>>>> >>> >> >> -- >> Best regards, >> Alexandr D.Sergeev >> _______________________________________________ >> freebsd-performance@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-performance >> To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd.org" >> > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd.org"