From owner-freebsd-stable@FreeBSD.ORG Tue Jun 8 09:11:50 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4F003106566C for ; Tue, 8 Jun 2010 09:11:50 +0000 (UTC) (envelope-from kasahara@nc.kyushu-u.ac.jp) Received: from elvenbow.cc.kyushu-u.ac.jp (unknown [IPv6:2001:200:905:1407:21b:21ff:fe52:5260]) by mx1.freebsd.org (Postfix) with ESMTP id CE2D18FC08 for ; Tue, 8 Jun 2010 09:11:49 +0000 (UTC) Received: from localhost (kasahara@localhost [IPv6:::1]) by elvenbow.cc.kyushu-u.ac.jp (8.14.4/8.14.4) with ESMTP id o589BlsW051847 for ; Tue, 8 Jun 2010 18:11:47 +0900 (JST) (envelope-from kasahara@nc.kyushu-u.ac.jp) Date: Tue, 08 Jun 2010 18:11:46 +0900 (JST) Message-Id: <20100608.181146.1224841629154014733.kasahara@nc.kyushu-u.ac.jp> To: freebsd-stable@freebsd.org From: Yoshiaki Kasahara In-Reply-To: <4BE82C5D.1080806@bit0.com> References: <4BE110E3.8040902@zirakzigil.org> <4BE82C5D.1080806@bit0.com> X-Fingerprint: CDA2 B6B6 6796 0DD3 9D80 2602 E909 4623 A15E A074 X-URL: http://rd.cc.kyushu-u.ac.jp/~kasahara/ X-Mailer: Mew version 7.0.50 on Emacs 24.0.50 / Mule 6.0 (HANACHIRUSATO) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: Re: Freebsd 8.0 kmem map too small X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jun 2010 09:11:50 -0000 Hello, I'd like to add another instance of similar problems. I recently updated my FreeBSD amd64 box with ZFS root and 8GB RAM from 8-STABLE (as of Mar 1st) to 8.1-PRERELEASE (as of May 27th). After that, my box started to crash every couple of days due to kmem_map too small. Here is a (last week) screenshot of Munin graph about the memory usage of the box: http://eron.info/munin-memory.png In "by month" graph, a white gap at the end of "Week 20" is the update period from 8-STABLE to 8.1-PRERELEASE I mentioned above. Before the upgrade, the system was rock solid without any kmem tuning in loader.conf (vm.kmem_size was around 2.7GB IIRC). After the update, I could see that more wired memory was assigned, and then steep drop (crash) occured. "by day" graph shows my experiment to bump vm.kmem_size=12G (recommended somewhere in this thread) and explicitly limit vfs.zfs.arc_max=2G. I was surprised because the wired memory quickly increased over 5GB... Then I noticed that the default value of vfs.zfs.zio.use_uma was 1 on amd64, so I turned it off and removed other memory tunings (kmem_size and arc_max) in loader.conf on Tuesday at 16:00 and rebooted. It seems that the usage of wired memory was stablized and no crash since then with the default kmem_size and arc_max. Does anyone have any idea about this behavior? My wild guess is that ZFS/UMA code is not stable on amd64 too, and it should be turned off by default for 8.1-RELEASE maybe.... Regards, -- Yoshiaki Kasahara Research Institute for Information Technology, Kyushu University kasahara@nc.kyushu-u.ac.jp