From owner-freebsd-fs@FreeBSD.ORG Tue May 11 17:10:45 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 228D9106566B for ; Tue, 11 May 2010 17:10:45 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta08.westchester.pa.mail.comcast.net (qmta08.westchester.pa.mail.comcast.net [76.96.62.80]) by mx1.freebsd.org (Postfix) with ESMTP id C5DDF8FC0A for ; Tue, 11 May 2010 17:10:43 +0000 (UTC) Received: from omta10.westchester.pa.mail.comcast.net ([76.96.62.28]) by qmta08.westchester.pa.mail.comcast.net with comcast id GBLk1e0070cZkys58HAkLD; Tue, 11 May 2010 17:10:44 +0000 Received: from koitsu.dyndns.org ([98.248.46.159]) by omta10.westchester.pa.mail.comcast.net with comcast id GHAi1e00P3S48mS3WHAjGW; Tue, 11 May 2010 17:10:43 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 41FB49B419; Tue, 11 May 2010 10:10:41 -0700 (PDT) Date: Tue, 11 May 2010 10:10:41 -0700 From: Jeremy Chadwick To: "Alexander V. Ribchansky" Message-ID: <20100511171041.GA2930@icarus.home.lan> References: <4BE95F1F.5090009@zk.informjust.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4BE95F1F.5090009@zk.informjust.ua> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-fs@freebsd.org Subject: Re: Freebsd 8.0 kmem map too small X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 May 2010 17:10:45 -0000 On Tue, May 11, 2010 at 04:43:59PM +0300, Alexander V. Ribchansky wrote: > As Artem advised, I comment out > > CFLAGS+=-DZIO_USE_UMA in sys/modules/zfs/Makefile > > and things like to be "back in USSR :)" if seriously - all become as good as before 18.04.2010 mega ZFS-MFC. > While with UMA, Wired memory constantly grow up to kmem_max limit and than PANIC! :(, without it, Wired is approx 380 - 450M on typical 8-STABLE KDE3 desktop with plain ZFS. > So or there is something wrong with my (and many many other's people's) hands or UMA broke ZFS at all. > > Thank you, Artem for hint! I already start to think, that revert to pre-18.04.2010 8-STABLE is the only solution. The interesting thing is that I'm not seeing this behaviour on either of our boxes running RELENG_8 kernel/world dated after 2010/04/18. Below are some stats. I don't know if these are any help. $ for i in localhost ra anubis; do echo ; ssh $i "uname -a ; sysctl kstat.zfs.misc.arcstats | egrep '(p|c|c_min|c_max|\.size|evict_skip|memory_throttle_count):' ; sysctl hw | egrep '(physmem|realmem|usermem):' ; egrep '(kmem_size|arc_max)=' /boot/loader.conf" ; done FreeBSD icarus.home.lan 8.0-STABLE FreeBSD 8.0-STABLE #0: Thu Apr 22 06:11:56 PDT 2010 root@icarus.home.lan:/usr/obj/usr/src/sys/X7SBA_RELENG_8_amd64 amd64 kstat.zfs.misc.arcstats.evict_skip: 0 kstat.zfs.misc.arcstats.p: 780308480 kstat.zfs.misc.arcstats.c: 1560281088 kstat.zfs.misc.arcstats.c_min: 201326592 kstat.zfs.misc.arcstats.c_max: 1610612736 kstat.zfs.misc.arcstats.size: 1560271336 kstat.zfs.misc.arcstats.memory_throttle_count: 0 hw.physmem: 4285079552 hw.usermem: 2546024448 hw.realmem: 5100273664 vm.kmem_size="2048M" vfs.zfs.arc_max="1536M" FreeBSD ra 8.0-STABLE FreeBSD 8.0-STABLE #0: Mon Apr 26 02:26:36 PDT 2010 root@ra:/usr/obj/usr/src/sys/X7SBI_RELENG_8_amd64 amd64 kstat.zfs.misc.arcstats.evict_skip: 211067 kstat.zfs.misc.arcstats.p: 2633281083 kstat.zfs.misc.arcstats.c: 2968678030 kstat.zfs.misc.arcstats.c_min: 469762048 kstat.zfs.misc.arcstats.c_max: 3758096384 kstat.zfs.misc.arcstats.size: 2622834360 kstat.zfs.misc.arcstats.memory_throttle_count: 0 hw.physmem: 8580050944 hw.usermem: 2859462656 hw.realmem: 9395240960 vm.kmem_size="4096M" vfs.zfs.arc_max="3584M" FreeBSD anubis 8.0-STABLE FreeBSD 8.0-STABLE #0: Fri Mar 26 10:41:14 PDT 2010 root@anubis:/usr/obj/usr/src/sys/PDSMI_PLUS_RELENG_8_amd64 amd64 kstat.zfs.misc.arcstats.evict_skip: 1395 kstat.zfs.misc.arcstats.p: 1879163904 kstat.zfs.misc.arcstats.c: 3758096384 kstat.zfs.misc.arcstats.c_min: 469762048 kstat.zfs.misc.arcstats.c_max: 3758096384 kstat.zfs.misc.arcstats.size: 1421240344 kstat.zfs.misc.arcstats.memory_throttle_count: 0 hw.physmem: 8580870144 hw.usermem: 6518259712 hw.realmem: 9126805504 vm.kmem_size="4096M" vfs.zfs.arc_max="3584M" -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB |