From owner-freebsd-current@FreeBSD.ORG Sun Oct 7 01:04:05 2007 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 498E916A420 for ; Sun, 7 Oct 2007 01:04:05 +0000 (UTC) (envelope-from joao.barros@gmail.com) Received: from nf-out-0910.google.com (nf-out-0910.google.com [64.233.182.191]) by mx1.freebsd.org (Postfix) with ESMTP id C6D6013C448 for ; Sun, 7 Oct 2007 01:04:04 +0000 (UTC) (envelope-from joao.barros@gmail.com) Received: by nf-out-0910.google.com with SMTP id b2so715457nfb for ; Sat, 06 Oct 2007 18:04:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; bh=GZLnOFG+79X7Ik1yDNaXcL1PDi9YRrslRdcXuAlxR0w=; b=hHglU8fIOlpo6cnvlAvK2CQtgoN5WuvTHx/QzaBBWTnbZMpi9Nbg7aGimgSsXi8OliYeyml9/vP1eNZ/pFN9wccMwnLmTvSj8oGWli+rQ0mjUJ5qskTdKxAP2OUqnyO79cRm15tJysQM1+KxnIKCa3X0fVuLFcP03pPkzgdk4gY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=GA4JXBSugEBqvU6nxxKfES7jrVP7Qa63Do3Jfz3EC/+b8KX2hoAlEjmfK+ToOVvl+QI6X5lmXiTxflFg4CMrQuGH46SbhPLvM5dfzIurOZwSrTcmJSY+pRLuITKkSjgssHW054MCLgZ5tLtLpgkzMTzxmTJ0jjbPB7teoDqtYlE= Received: by 10.78.180.18 with SMTP id c18mr9816666huf.1191719043101; Sat, 06 Oct 2007 18:04:03 -0700 (PDT) Received: by 10.78.163.2 with HTTP; Sat, 6 Oct 2007 18:04:03 -0700 (PDT) Message-ID: <70e8236f0710061804o62ea85c6k1d8a5e7d3600ef15@mail.gmail.com> Date: Sun, 7 Oct 2007 02:04:03 +0100 From: "Joao Barros" To: "Pawel Jakub Dawidek" In-Reply-To: <20071005000046.GC92272@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <20071005000046.GC92272@garage.freebsd.pl> Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org Subject: Re: ZFS kmem_map too small. X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Oct 2007 01:04:05 -0000 On 10/5/07, Pawel Jakub Dawidek wrote: > Hi. > > We'are about to branch RELENG_7 and I'd like to start discussion with > folks that experience 'kmem_map too small' panic with the latest HEAD. > > I'm trying hard to reproduce it and I can't, so I need to gather more > info how you are able to provoke this panic. > > What I did was to rsync 200 FreeBSD src trees from one directory to > another on the same ZFS file system. It worked fine. > > The system I'm using is i386 and the only tuning I did is bigger > kmem_map. From my /boot/loader.conf: > > vm.kmem_size=629145600 > vm.kmem_size_max=629145600 > > The machine is dual core Pentium D 3GHz with 1GB of RAM. My pool is: > > lcf:root:/tank/0# zpool status > pool: tank > state: ONLINE > scrub: none requested > config: > > NAME STATE READ WRITE CKSUM > tank ONLINE 0 0 0 > ad4 ONLINE 0 0 0 > ad5 ONLINE 0 0 0 > ad6 ONLINE 0 0 0 > ad7 ONLINE 0 0 0 > > errors: No known data errors > > If you can still see those panic, please let me know as soon as possible > and try to describe what your workload looks like, how to reproduce it, > etc. I'd really like ZFS to be rock-stable for 7.0 even on i386. > i386 with 1GB here. I used to get this when chown'ing some thousand files recursively via ssh. Last time I got this was unraring 2GB files from n x 95MB rars via NFS. My system: xeon# zpool status pool: r4x320 state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM r4x320 ONLINE 0 0 0 raidz1 ONLINE 0 0 0 ad0s1d ONLINE 0 0 0 ad1s1d ONLINE 0 0 0 ad4 ONLINE 0 0 0 ad6 ONLINE 0 0 0 errors: No known data errors xeon# cat /boot/loader.conf zfs_load="YES" vfs.root.mountfrom="zfs:r4x320" vfs.zfs.prefetch_disable=1 (I have this to improve video play) xeon# sysctl vm | grep kmem vm.kmem_size_scale: 3 vm.kmem_size_max: 335544320 vm.kmem_size_min: 0 vm.kmem_size: 335544320 xeon# sysctl -a | grep vnodes kern.maxvnodes: 52242 kern.minvnodes: 17414 vfs.freevnodes: 7797 vfs.wantfreevnodes: 17414 vfs.numvnodes: 8230 I usually set kern.maxvnodes to 50000 manually and everything is ok but I see that I forgot to on my last reboot and haven't seen any problems yet: xeon# uptime 1:56AM up 4 days, 15:06 off-topic: it's lovely not having to have your 874GB fs with millions of files checked after a crash or power failure :-D -- Joao Barros