From owner-freebsd-stable Mon Feb 4 4:28:57 2002 Delivered-To: freebsd-stable@freebsd.org Received: from dream.mplik.ru (dream.mplik.ru [195.58.1.132]) by hub.freebsd.org (Postfix) with ESMTP id ABE4937B417 for ; Mon, 4 Feb 2002 04:28:49 -0800 (PST) Received: from sight (sight.mplik.ru [195.58.27.104]) by dream.mplik.ru (8.9.3/8.9.1) with ESMTP id RAA94875; Mon, 4 Feb 2002 17:28:36 +0500 (YEKT) Date: Mon, 4 Feb 2002 17:28:33 +0500 From: Sergey Gershtein X-Mailer: The Bat! (v1.53bis) Business Reply-To: Sergey Gershtein Organization: Ural Relcom Ltd X-Priority: 3 (Normal) Message-ID: <114283707399.20020204172833@ur.ru> To: Peter Jeremy Cc: freebsd-stable@FreeBSD.ORG Subject: FS gurus needed! (was: Strange lock-ups during backup over nfs after adding 1024M RAM) In-Reply-To: <20020204130730.B72285@gsmx07.alcatel.com.au> References: <20020126204941.H17540-100000@resnet.uoregon.edu> <1931130530386.20020128130947@ur.ru> <20020130073449.B78919@gsmx07.alcatel.com.au> <791310002584.20020130150111@ur.ru> <20020131111153.Y72285@gsmx07.alcatel.com.au> <1427021336.20020201123650@ur.ru> <20020204130730.B72285@gsmx07.alcatel.com.au> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-stable@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Monday, February 04, 2002 Peter Jeremy wrote: PJ> On 2002-Feb-01 12:36:50 +0500, Sergey Gershtein wrote: >>Here's what "vmstat -m" says about "FFS node": >> >>Memory statistics by type Type Kern >> Type InUse MemUse HighUse Limit Requests Limit Limit Size(s) >> ... >> FFS node152293 76147K 76479K102400K 3126467 0 0 512 >> ... PJ> One oddity here is the Size - "FFS node" is used to allocate struct PJ> inode's and they should be 256 bytes on i386. Are you using something PJ> other than an i386 architecture? Unless this is a cut-and-paste PJ> error, I suspect something is radically wrong with your kernel. Yes, it's i386 and it's not cut-and-paste error. The current output of vmstat -m says: ... FFS node152725 76363K 76479K102400K 9247602 0 0 512 ... vfscache157865 10671K 11539K102400K 9668497 0 0 64,128,256,512,512K ... The system uptime is 5 days, backup is temporarily disabled. I put the coplete output of 'vmstat -m', some other commands and kernel config on the web on http://storm.mplik.ru/fbsd-stable/ so you can have a look at it. By the way, on our second server running the same hardware the size of "FFS node" is also 512. How can it be so? PJ> By default, the memory limit is 1/2 vm_kmem_size, which is 1/3 physical PJ> memory, capped to 200MB. Which means you've hit the default cap. PJ> You can increase this limit with the loader environment PJ> kern.vm.kmem.size (see loader(8) for details). (This is also capped PJ> at twice the physical memory - which won't affect you). Before you go PJ> overboard increasing this, note that the kernel virtual address space PJ> is only 1GB. Hmm. Not sure what to do. Shell I try to play with kern.vm.kmem.size or better not touch it? I am now thinking that removing the extra memory we've added is the best solution to the problem. I don't like this solution though. PJ> How many open files do you expect on your box? PJ> Is it reasonable for there to be >>150,000 active inodes? ptat -T right now says: 666/4096 files 0M/511M swap space I don't expect the number of open files go beyond 1,000-1,500. The only problem is accessing a lot (more than a 1,000,000) of small files over NFS. But if I understand correctly, those files should be opened and closed one by one, not all together. Is that right? PJ> Does "vfscache" have around the same number of InUse entries as "FFS node"? Yes, it seems so (see above). What does it mean? PJ> What is the output of "sysctl vfs"? See http://storm.mplik.ru/fbsd-stable/sysctl_vfs.txt PJ> PS: I'm still hoping that one of the FS gurus will step in and point PJ> out what's wrong. I changed the subject of my message to catch attention of FS gurus on the list. Thank you, Sergey To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-stable" in the body of the message