From owner-freebsd-stable Mon Feb 4 13:30:14 2002 Delivered-To: freebsd-stable@freebsd.org Received: from netau1.alcanet.com.au (ntp.alcanet.com.au [203.62.196.27]) by hub.freebsd.org (Postfix) with ESMTP id 6318337B426 for ; Mon, 4 Feb 2002 13:30:05 -0800 (PST) Received: from mfg1.cim.alcatel.com.au (mfg1.cim.alcatel.com.au [139.188.23.1]) by netau1.alcanet.com.au (8.9.3 (PHNE_22672)/8.9.3) with ESMTP id IAA14780; Tue, 5 Feb 2002 08:30:03 +1100 (EDT) Received: from gsmx07.alcatel.com.au by cim.alcatel.com.au (PMDF V5.2-32 #37640) with ESMTP id <01KDWPWYRDQOVMIHVG@cim.alcatel.com.au>; Tue, 5 Feb 2002 08:30:02 +1100 Received: (from jeremyp@localhost) by gsmx07.alcatel.com.au (8.11.6/8.11.6) id g14LU0T86081; Tue, 05 Feb 2002 08:30:00 +1100 Content-return: prohibited Date: Tue, 05 Feb 2002 08:30:00 +1100 From: Peter Jeremy Subject: Re: FS gurus needed! (was: Strange lock-ups during backup over nfs after adding 1024M RAM) In-reply-to: <114283707399.20020204172833@ur.ru>; from sg@ur.ru on Mon, Feb 04, 2002 at 05:28:33PM +0500 To: Sergey Gershtein Cc: freebsd-stable@FreeBSD.ORG Mail-Followup-To: Sergey Gershtein , freebsd-stable@FreeBSD.ORG Message-id: <20020205083000.I72285@gsmx07.alcatel.com.au> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-disposition: inline User-Agent: Mutt/1.2.5i References: <20020126204941.H17540-100000@resnet.uoregon.edu> <1931130530386.20020128130947@ur.ru> <20020130073449.B78919@gsmx07.alcatel.com.au> <791310002584.20020130150111@ur.ru> <20020131111153.Y72285@gsmx07.alcatel.com.au> <1427021336.20020201123650@ur.ru> <20020204130730.B72285@gsmx07.alcatel.com.au> <114283707399.20020204172833@ur.ru> Sender: owner-freebsd-stable@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 2002-Feb-04 17:28:33 +0500, Sergey Gershtein wrote: >On Monday, February 04, 2002 Peter Jeremy wrote: >PJ> One oddity here is the Size - "FFS node" is used to allocate struct >PJ> inode's and they should be 256 bytes on i386. Are you using something >PJ> other than an i386 architecture? Unless this is a cut-and-paste >PJ> error, I suspect something is radically wrong with your kernel. > >Yes, it's i386 and it's not cut-and-paste error. Looking at your dmesg, you're running RELENG_4_4 - I has incorrectly assumed you were running RELENG_4. The size of struct inode has changed (it shrank by 4 bytes), meaning it's now 256 bytes instead of 260 bytes (which requires 512 bytes from the allocator). That explains that discrepancy. This is corrected in src/sys/ufs/ufs/inode.h,v 1.28.2.2. I think RELENG_4_4 also explains your problem: There was a problem that vnodes were being cached too long and there was effectively no upper limit on their number. I gather that under some conditions (which I don't recall, but you seem to have hit) the vnode cache (and underlying inode cache) would eat all the KVM. There were a variety of commits by Matt Dillon between October and early November that fixed this. Have a search through the archives for references to "kern.maxvnodes". I'd suggest that you look at moving to RELENG_4_5 or RELENG_4. This will halve the size of FFS node entries (which is most of your kernel memory) and allow you to limit it to something saner. >Hmm. Not sure what to do. Shell I try to play with kern.vm.kmem.size >or better not touch it? I'm not sure why there is a 200MB cap. Since your machine isn't short of physical memory (no swapping), I don't see any problem with increasing it to something like 500MB (which would be its uncapped value). Try adding "kern.vm.kmem.size=524288000" to /boot/loader.conf > I am now thinking that removing the extra >memory we've added is the best solution to the problem. I don't like >this solution though. I don't understand why you don't have the problem on the smaller box. >I don't expect the number of open files go beyond 1,000-1,500. The >only problem is accessing a lot (more than a 1,000,000) of small files >over NFS. But if I understand correctly, those files should be opened >and closed one by one, not all together. Is that right? The problem seems to be the vnode cache - which isn't being cleaned correctly. >PJ> Does "vfscache" have around the same number of InUse entries as "FFS node"? > >Yes, it seems so (see above). What does it mean? Most (all?) "FFS node" entries should be pointed to by vfscache (vnode) entries. If this isn't so, then there's a bug releasing inodes. Peter To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-stable" in the body of the message