From owner-freebsd-arch@FreeBSD.ORG Sat Nov 8 12:59:50 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B68E516A4CE for ; Sat, 8 Nov 2003 12:59:50 -0800 (PST) Received: from smtp.omnis.com (smtp.omnis.com [216.239.128.26]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0981B43FF7 for ; Sat, 8 Nov 2003 12:59:50 -0800 (PST) (envelope-from wes@softweyr.com) Received: from softweyr.homeunix.net (66-91-236-204.san.rr.com [66.91.236.204]) by smtp-relay.omnis.com (Postfix) with ESMTP id 2B18B5B625; Sat, 8 Nov 2003 12:53:22 -0800 (PST) From: Wes Peters Organization: Softweyr To: Bruce Evans Date: Sat, 8 Nov 2003 12:59:48 -0800 User-Agent: KMail/1.5.4 References: <200311041737.20467.wes@softweyr.com> <200311061652.48948.wes@softweyr.com> <20031107211239.O3343@gamplex.bde.org> In-Reply-To: <20031107211239.O3343@gamplex.bde.org> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200311081259.48214.wes@softweyr.com> cc: arch@FreeBSD.org Subject: Re: newfs and mount vs. half-baked disks X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Nov 2003 20:59:50 -0000 On Friday 07 November 2003 03:01 am, Bruce Evans wrote: > On Thu, 6 Nov 2003, Wes Peters wrote: > > On Wednesday 05 November 2003 03:18, Bruce Evans wrote: > > > On Tue, 4 Nov 2003, Wes Peters wrote: > > > > I emailed Kirk about this state of affairs and he confirmed that > > > > newfs was developed with operator intervention in mind. He > > > > suggested employing one of the unused flags in the filesystem > > > > header as a 'consistent' flag, setting it to 'not consistent' at > > > > the beginning of newfs, and then updating to 'is consistent' at > > > > the end. The performance hit in updating all superblock copies > > > > at the end is small but noticable (< 1s on a rather slow 6GB > > > > filesystem). > > > > > > There is no need to use a new flag. Just set the magic number to a > > > value different from both FS_UFS1_MAGIC and FS_UFS2_MAGIC, e.g., to > > > 0, until newfs is nearly finished. > > > > I specifically don't want to do that because I want the state > > "interrupted newfs operation" to be discernable from the state > > "something stomped on your superblock." This I believe better shows > > that the superblock is valid but the filesystem is not (yet). > > That's not a very useful distinction, especially for your application > where the disk contents is disposable and you re-newfs it a lot. Oh, but after the newfs we take different actions depending on whether we think we're creating a new filesystem or restoring an old one. In particular, if we are restoring a previous filesystem, there are references to restored data that need to be checked for validity. None of these references exist on a new system, so we can avoid the cost of searching for them. > After > a crash the complete state (as given by the disk contents) may be > almost anywhwere between its initial and final values, depending on > buffering etc., so it would be very difficult just to determine what > it is if you needed it. > > Some cases can be discerned anyway, depending on how much erasing of > metadata newfs does when it starts. E.g., if there was an ffs file > system on the disk, then this will be recorded in the disk label unless > that feature has been axed). newfs doesn't rewrite the label until > just before it finishes, so the old label can be looked at to determine > what was there. Writing the label last may be a bug though. Perhaps > newfs should erase all the primary metadata for the old filesystem > when it starts so as to minimise the window where there may be > conflicting metadata. > > Clearing the magic number works better because it requires no kernel > changes and no changes to applications other than ufs. In particular, > half-baked file systems formatted with the changed newfs work right > under all versions of FreeBSD (i.e., they don't work and don't cause > panics), and utilities like dumpfs and fsdb don't need to be changed > to display and/or edit the newly used field. Actually, changing the magic number to a differing non-zero magic value has all the benefits you suggest, plus the benefit of signifying what I'm looking for (except for a tiny window of vulnerability where a disk *could* have the alternate magic number written in the superblock location by chance.) > > The name fs_state suggests someone was thinking of recording some > > sort of state in here that was never implemented. I've simply used > > it to record states 'newfs operation completed' and 'newfs operation > > not completed.' > > Its comment also suggests that it was for validating fs_clean. This > goes back to at least FreeBSD-1 where both fs_state and fs_clean are > unused. fs_clean wasn't used in 4.4BSD-Lite1 either. Use of it was > added in FreeBSD and/or NetBSD and picked up by Lite2. I am nervous > about using yet another variable for things related to state. There > is a large enough mess for fs_clean already. We now also have > FS_UNCLEAN in fs_flags (which were also unused until relatively > recently). As I understand it (the details were simpler when I > understood them all), FS_UNCLEAN was originally fs_clean done right > except it probably belongs in fs_state. It was just fs_clean inverted > and put in a bitmap. Now FS_UNCLEAN is tangled up with FS_NEEDSFSCK > and is not simply fs_clean inverted. Yes, trying to understand the interactions of fs_state and FS_UNCLEAN made my head hurt. Badly. I'll do a bit of testing with the alternate magic number and report what I observe, but probably not until Monday. I left my testing disk at work and I have too much sysadmin work at home to work on code this weekend. Sigh. -- Where am I, and what am I doing in this handbasket? Wes Peters wes@softweyr.com