From owner-freebsd-fs@FreeBSD.ORG Mon Oct 5 07:24:19 2009 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EF439106568F; Mon, 5 Oct 2009 07:24:18 +0000 (UTC) (envelope-from bra@fsn.hu) Received: from people.fsn.hu (people.fsn.hu [195.228.252.137]) by mx1.freebsd.org (Postfix) with ESMTP id 057CA8FC15; Mon, 5 Oct 2009 07:24:17 +0000 (UTC) Received: by people.fsn.hu (Postfix, from userid 1001) id 79DEC1371A2; Mon, 5 Oct 2009 09:24:16 +0200 (CEST) X-CRM114-Version: 20090423-BlameSteveJobs ( TRE 0.7.6 (BSD) ) MF-ACE0E1EA [pR: 17.0389] X-CRM114-CacheID: sfid-20091005_09241_A1B3987E X-CRM114-Status: Good ( pR: 17.0389 ) Message-ID: <4AC99F1D.3040300@fsn.hu> Date: Mon, 05 Oct 2009 09:24:13 +0200 From: Attila Nagy User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.23) Gecko/20090817 Thunderbird/2.0.0.23 Mnenhy/0.7.6.0 MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <4AC1E540.9070001@fsn.hu> <4AC5B2C7.2000200@fsn.hu> <20091002184526.GA1660@garage.freebsd.pl> In-Reply-To: <20091002184526.GA1660@garage.freebsd.pl> X-Stationery: 0.4.10 Content-Type: text/plain; charset=ISO-8859-2; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.3 (people.fsn.hu); Mon, 05 Oct 2009 09:24:15 +0200 (CEST) Cc: freebsd-fs@FreeBSD.org Subject: Re: ARC size constantly shrinks, then ZFS slows down extremely X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Oct 2009 07:24:19 -0000 On 10/02/09 20:45, Pawel Jakub Dawidek wrote: > On Fri, Oct 02, 2009 at 09:59:03AM +0200, Attila Nagy wrote: > >> Backing out this change from the 8-STABLE kernel: >> http://svn.freebsd.org/viewvc/base/head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c?r1=191901&r2=191902 >> >> makes it survive about half and hour of IMAP searching. Of course only >> time will tell whether this helps in the long run, but so far 10/10 >> tries succeeded to kill the machine with this method... >> > > Could you try this patch: > > http://people.freebsd.org/~pjd/patches/arc.c.4.patch > Sure. But before that, a report with the above modification: the machine has survived some days, then started to behave strangely. Meaning I could ping it, I could log in to the IMAP service (running from ZFS), read some mails, but not all. I could not access it via ssh (which runs from UFS), but an already running top from a different session was alive. It showed: last pid: 11272; load averages: 0.00, 0.00, 0.00 up 3+15:21:13 09:11:43 149 processes: 1 running, 143 sleeping, 1 zombie, 4 waiting CPU: 0.0% user, 0.0% nice, 0.2% system, 0.0% interrupt, 99.8% idle Mem: 234M Active, 197M Inact, 559M Wired, 111M Buf, 440K Free Swap: 4096M Total, 976K Used, 4095M Free PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 78492 root 1 44 0 4700K 2156K CPU1 1 5:37 0.00% top 92343 root 1 44 0 4132K 1576K nanslp 1 4:12 0.00% gstat 13401 root 1 44 0 1528K 456K piperd 0 2:19 0.00% readproctitl 12679 root 1 44 0 3932K 1236K vmwait 1 2:12 0.00% zpool 35988 125 4 45 0 16892K 5968K sigwai 0 1:53 0.00% milter-greyl 25656 root 1 45 0 1536K 564K getblk 0 1:45 0.00% supervise 25798 root 1 44 0 1536K 564K vmwait 0 1:44 0.00% supervise 28406 root 1 44 0 1536K 544K vmwait 0 1:43 0.00% supervise 30226 root 1 44 0 1536K 544K vmwait 0 1:43 0.00% supervise 35401 root 1 44 0 1536K 544K vmwait 0 1:42 0.00% supervise 29203 root 1 44 0 1536K 544K vmwait 0 1:42 0.00% supervise 21629 389 6 44 0 91664K 41892K ucond 0 1:02 0.00% slapd 72283 60 1 44 0 80972K 1948K select 1 0:34 0.00% idled 98960 root 1 44 0 9396K 2544K select 1 0:32 0.00% sshd 1550 root 1 44 0 3340K 940K vmwait 1 0:32 0.00% syslogd 5463 125 1 44 0 6924K 2036K vmwait 0 0:27 0.00% qmgr 54193 root 1 44 0 9396K 2516K select 0 0:22 0.00% sshd I could not log into the console, it didn't even gave a "user name" filed after hitting enter. Strange. I will try the patch.