Date: Tue, 8 Jun 2010 02:54:44 -0700 From: Jeremy Chadwick <freebsd@jdc.parodius.com> To: Yoshiaki Kasahara <kasahara@nc.kyushu-u.ac.jp> Cc: pjd@freebsd.org, freebsd-stable@freebsd.org Subject: Re: Freebsd 8.0 kmem map too small Message-ID: <20100608095444.GA86953@icarus.home.lan> In-Reply-To: <20100608.181146.1224841629154014733.kasahara@nc.kyushu-u.ac.jp> References: <4BE110E3.8040902@zirakzigil.org> <q2vb269bc571005050819nde819098vfd3306170639a9c9@mail.gmail.com> <4BE82C5D.1080806@bit0.com> <20100608.181146.1224841629154014733.kasahara@nc.kyushu-u.ac.jp>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, Jun 08, 2010 at 06:11:46PM +0900, Yoshiaki Kasahara wrote: > Hello, > > I'd like to add another instance of similar problems. I recently > updated my FreeBSD amd64 box with ZFS root and 8GB RAM from 8-STABLE > (as of Mar 1st) to 8.1-PRERELEASE (as of May 27th). After that, my > box started to crash every couple of days due to kmem_map too small. > > Here is a (last week) screenshot of Munin graph about the memory usage > of the box: > > http://eron.info/munin-memory.png > > In "by month" graph, a white gap at the end of "Week 20" is the update > period from 8-STABLE to 8.1-PRERELEASE I mentioned above. Before the > upgrade, the system was rock solid without any kmem tuning in > loader.conf (vm.kmem_size was around 2.7GB IIRC). > > After the update, I could see that more wired memory was assigned, and > then steep drop (crash) occured. > > "by day" graph shows my experiment to bump vm.kmem_size=12G > (recommended somewhere in this thread) and explicitly limit > vfs.zfs.arc_max=2G. I was surprised because the wired memory quickly > increased over 5GB... > > Then I noticed that the default value of vfs.zfs.zio.use_uma was 1 on > amd64, so I turned it off and removed other memory tunings (kmem_size > and arc_max) in loader.conf on Tuesday at 16:00 and rebooted. It > seems that the usage of wired memory was stablized and no crash since > then with the default kmem_size and arc_max. > > Does anyone have any idea about this behavior? > > My wild guess is that ZFS/UMA code is not stable on amd64 too, and it > should be turned off by default for 8.1-RELEASE maybe.... I realise you're talking about amd64, but I'm not sure how UMA is getting enabled on i386 to begin with. It does look like it's enabled on amd64 by default. src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c -- #if defined(__amd64__) static int zio_use_uma = 1; #else static int zio_use_uma = 0; #endif SYSCTL_DECL(_vfs_zfs); SYSCTL_NODE(_vfs_zfs, OID_AUTO, zio, CTLFLAG_RW, 0, "ZFS ZIO"); TUNABLE_INT("vfs.zfs.zio.use_uma", &zio_use_uma); SYSCTL_INT(_vfs_zfs_zio, OID_AUTO, use_uma, CTLFLAG_RDTUN, &zio_use_uma, 0, "Use uma(9) for ZIO allocations"); Also note that this tunable does not appear in sysctl -a output; I believe this is caused by the first parameter to SYSCTL_INT() being incorrect -- should be _vfs_zfs, not _vfs_zfs_zio. Regardless of that, it's still adjustable in loader.conf. -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB |
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20100608095444.GA86953>