From owner-freebsd-performance@FreeBSD.ORG Tue Mar 15 15:30:44 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C3F7216A4CE for ; Tue, 15 Mar 2005 15:30:44 +0000 (GMT) Received: from mr01.hansenet.de (mr01.hansenet.de [213.191.74.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5BAC143D1F for ; Tue, 15 Mar 2005 15:30:44 +0000 (GMT) (envelope-from db@nipsi.home.net) Received: from nipsi.home.net (213.39.140.190) by mr01.hansenet.de (6.7.010) id 422EC055000276E2 for freebsd-performance@freebsd.org; Tue, 15 Mar 2005 16:30:42 +0100 Received: from localhost (localhost [127.0.0.1]) by nipsi.home.net (Postfix) with ESMTP id A5C2E1E6F3 for ; Tue, 15 Mar 2005 16:32:11 +0100 (CET) Received: from nipsi.home.net ([127.0.0.1]) by localhost (nipsi.home.net [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 07721-09 for ; Tue, 15 Mar 2005 16:32:09 +0100 (CET) Received: by nipsi.home.net (Postfix, from userid 1001) id D1AE31E701; Tue, 15 Mar 2005 16:32:09 +0100 (CET) Date: Tue, 15 Mar 2005 16:32:09 +0100 From: Dennis Berger To: freebsd-performance@freebsd.org Message-ID: <20050315153209.GA8863@nipsi.home.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.6i X-Virus-Scanned: amavisd-new at ARRAY(0x87d7e14) Subject: vfs.ufs.dirhash_maxmem is a bit low X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Mar 2005 15:30:44 -0000 hi all, I recently had the problem to change a line in about 2.5 million files. In about 1530 subdirectories. i started this command. find . \( -not -name restoresym\* -and -not -name \*.log -and -not -name \*.gif \) -type f -print0 | xargs -0 sed -i '' 's/\/wurstbrot\//\/kaesebrot\//' on another shell i watched with gstat how it works after several minutes i noticed a terrible breakdown. no read/write anymore. After watching with top i notice 25% system usage. (it's a 4 processor mashine) I tracked down the error to the vfs.ufs.dirhash. I guess the system is running out of memory for the hashtable so everything is getting terrible slow. I raised vfs.ufs.dirhash_maxmem to 30 mb. after that everything was fine again. My request can we add white kernel message for the behavier of running out of hashmemory? Like it is for running out of open files. A message will appear on /dev/console indicating that the dirhash is too low. Or pointing out in tuning(9) that it should raised of servers with lots of files. Or raised the default from 2MB to 10MB or something. any comments? ragards, -Dennis