From owner-freebsd-fs@FreeBSD.ORG Thu Jul 31 01:41:19 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B196937B401 for ; Thu, 31 Jul 2003 01:41:19 -0700 (PDT) Received: from bluejay.mail.pas.earthlink.net (bluejay.mail.pas.earthlink.net [207.217.120.218]) by mx1.FreeBSD.org (Postfix) with ESMTP id 209BF43F75 for ; Thu, 31 Jul 2003 01:41:19 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from user-38lc0sl.dialup.mindspring.com ([209.86.3.149] helo=mindspring.com) by bluejay.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 19i8zt-0005Vi-00; Thu, 31 Jul 2003 01:41:18 -0700 Message-ID: <3F28D5F5.2B07551B@mindspring.com> Date: Thu, 31 Jul 2003 01:40:21 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Brooks Davis References: <20030730183307.GA11345@Odin.AC.HMC.Edu> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a4201be0800d0aafde4977e2a3f783f9f8a2d4e88014a4647c350badd9bab72f9c350badd9bab72f9c cc: fs@freebsd.org Subject: Re: fsck memory usage X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Jul 2003 08:41:20 -0000 Brooks Davis wrote: > Does anyone have a formula to give fsck memory requirements based on > disk size and newfs arguments. I'm going to be building some large > (3.3TB) file systems and I'd like to figure out if I'm going to need > to go 64-bit to have sufficent address space to fsck them. The mean > filesize will be pretty big so I will be able to set bytes/i-node > fairly high. The amount of memory really depends on which passes you end up having to run, since it has to "remember" state information. By far, the thing that takes the most time is the cylinder group bitmaps, which it has to assume are dirty, and then correlate all the blocks in every file on the disk vs. the bitmaps to make sure that a particular bit which is set is OK to unset. This is also the pass that takes the most memory, in my own experience, unless you have a single very very large file that lost its directory entry but has a referenced inode, and you're moving it to lost+found. About 6 months ago I suggested a change to the on disk layout and the addition of a soft dependency, so you you could (1) know which CG bitmaps are dirty and (2) put in a concurrency stall point to force CG bitmaps out to disk, once they hit the high water mark. You would still have to pass through the entire directory stucture once in order to clean the CG bitmaps, but with a smaller list of bitmaps, there would be a heck of a lot less memory (and swap) that was consumed in the process. For this to work, you have to have a "dirty CG bitmap list"; my initial feeling was that you could put a magic number in inode 1 (used to hold whiteout information; formerly used to hold the bad block list) and steal a fixed size block from the front of it, and still maintain your whiteouts. In any case, the answer to "how much memory" really depends on (1) how many CG's you have, (2) the size of a single bitmap, (3) the largest file you have on your disk at one time, and (4) the amount of "fill" that your directories have. Basically, any one of these could, if it were "just right", push the number higher than anoy of the other pushes it. So basically it's a "highest tide" function, if you want to calculate how much it's going to take for a given instance, and it's "really, really a lot", if you don't limit the size of a single file on the volume, and you just want to calculate out the max it will take. Any way you look at it, though, the answer will also vary with wersion of fsck (there's been some internal reorganization at various points in the past), so you're not going to be able to make up a number that applies to all BSD's. -- Terry