From owner-freebsd-current@FreeBSD.ORG Wed May 26 14:48:59 2004 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E4F4816A4CE for ; Wed, 26 May 2004 14:48:59 -0700 (PDT) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.126.189]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1E9FB43D46 for ; Wed, 26 May 2004 14:48:59 -0700 (PDT) (envelope-from se@freebsd.org) Received: from [212.227.126.208] (helo=mrelayng.kundenserver.de) by moutng.kundenserver.de with esmtp (Exim 3.35 #1) id 1BT6GQ-0000xt-00; Wed, 26 May 2004 23:48:42 +0200 Received: from [80.132.239.237] (helo=Gatekeeper.FreeBSD.org) by mrelayng.kundenserver.de with asmtp (Exim 3.35 #1) id 1BT6GP-0001Ee-00; Wed, 26 May 2004 23:48:41 +0200 Received: from StefanEsser.FreeBSD.org (StefanEsser [192.168.0.10]) by Gatekeeper.FreeBSD.org (Postfix) with ESMTP id D5037612C; Wed, 26 May 2004 23:48:22 +0200 (CEST) Received: by StefanEsser.FreeBSD.org (Postfix, from userid 200) id 5D7DE2314; Wed, 26 May 2004 23:48:22 +0200 (CEST) Date: Wed, 26 May 2004 23:48:22 +0200 From: Stefan =?iso-8859-1?Q?E=DFer?= To: Ivan Voras Message-ID: <20040526214822.GA12171@StefanEsser.FreeBSD.org> Mail-Followup-To: Stefan =?iso-8859-1?Q?E=DFer?= , Ivan Voras , current@freebsd.org References: <40B4ECC8.50808@fer.hr> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <40B4ECC8.50808@fer.hr> User-Agent: Mutt/1.5.6i X-Provags-ID: kundenserver.de abuse@kundenserver.de auth:fa3fae9b6ca38d745862a668565919f6 cc: current@freebsd.org Subject: Re: Softupdates a mount option? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 May 2004 21:49:00 -0000 On 2004-05-26 21:15 +0200, Ivan Voras wrote: > This has been really nagging me for a long time: Why aren't softupdates > made a mount option (like 'sync' and 'async')? Do I remember correctly that > it is done so in NetBSD (where it's called softdeps), so it's doable? I prepared and posted patches to make soft updates a mount option several years ago (June 2000, in fact). I'll just append the message I received from Kirk McKusick to the end of this Mail. In case you are interested, I'll prepare a patch against a recent -current, tomorrow (the diff I add to this mail is as of 4 years ago). I'd be happy to commit my diffs (which I have kept in my system, all the years) to -current, especially if NetBSD got that mount option 3 years ago. Regards, STefan >>From mckusick@mckusick.com Sat Jul 1 13:49:15 2000 Received: by hub.freebsd.org (Postfix) id CB4A437B9B7; Wed, 28 Jun 2000 13:51:08 -0700 (PDT) Delivered-To: se@freebsd.org Received: from beastie.mckusick.com (beastie.mckusick.com [209.31.233.184]) by hub.freebsd.org (Postfix) with ESMTP id 8134937C1C5; Wed, 28 Jun 2000 13:51:01 -0700 (PDT) (envelope-from mckusick@mckusick.com) Received: from beastie.mckusick.com (localhost [127.0.0.1]) by beastie.mckusick.com (8.9.3/8.9.3) with ESMTP id NAA05776; Wed, 28 Jun 2000 13:51:01 -0700 (PDT) (envelope-from mckusick@beastie.mckusick.com) Message-Id: <200006282051.NAA05776@beastie.mckusick.com> To: arch@freebsd.org Subject: Re: Soft updates mount change Cc: Stefan Esser Date: Wed, 28 Jun 2000 13:51:00 -0700 From: Kirk McKusick Content-Type: text Status: RO Content-Length: 12566 Lines: 301 After a long discussion (see below) with Stefan Esser, I have agreed that it is reasonable to make soft updates a mount option once we flip over to making them on by default. If you do not want soft updates, you will add the option `nosoftdep' to your fstab entry. That makes the extra options in /etc/fstab the exception rather than the rule and generally does not require naive administrators to know about them. It also avoids having a flag day when all the entries in /etc/fstab need to be flipped over as adding them now would do. I do not anticipate making them default on until they have had wider operational experience from the new licensing and until I resolve the problems with using them on nearly full filesystems (writes failing because space has not yet been freed from recently deleted files). Kirk McKusick =-=-=-=-=-=-=-= Date: Wed, 28 Jun 2000 00:38:24 +0200 From: Stefan Esser To: Kirk McKusick Cc: Stefan Esser , Stefan.Esser@o-tel-o.de Subject: Re: Soft updates mount change Reply-To: Stefan Esser On 2000-06-24 12:00 -0700, Kirk McKusick wrote: Please grant me a last chance to state my reasons, why I think that soft-updates might become a mount option: 1) The pre-dominant reason to store the soft-updates status in the super-block is for FSCK. 2) File system update policy decisions are per file-system, not per medium. 3) It should not be neccessary to have single-user console access to change the soft-updates enable bit. (Yes, I know about tunefs on a partition that is mounted R/O, but that will not work in all situations.) Regarding 1: There is a requirement, that fsck is able to determine whether the file-system was previously mounted with or with soft-updates. This asks for a way to keep that information in a place where it can't be deleted by accident, if the file system was not unmounted in a clean state. Tunefs operates on unmounted file-systems, whether clean or not. That means, that the SOFTDEP but in fs_flags can be modified either way on an unclean file system, as long as it is not mounted. The decision to use soft-updates should not share a bit with the status flag that controls fsck's operation. Regarding 2: My policy to use or not use soft-updates depends on the file-system, not the partition or medium (consider removable media). How about a ZIP drive that is used with soft-updates in my system, but is written to and disconnected without a clean unmount on a system, that does not have soft-updates in the kernel (e.g. FreeBSD-2.2.8). The medium will require a fsck on my system, and fsck must assume it was last written to with soft-update dependencies enforced, when in reality it was not. This is a constructed case (I do not even own a ZIP drive), but meant to explain my point of view, that the use of soft-updates does not depend on the medium but rather on the file system. My policy currently states, that soft-updates be used on all file systems except "/" (since I had bad luck with /bin being only partly populated after a "make installworld", which had required three times the free space required by the new binaries). I might have a policy, that requires /var to be written "sync", which kind of conflicts with soft-updates. I may have removable media that are moved between a number of systems and mounted on different mount points, with changing access and usage policies incl. whether async, sync or soft-updates are suitable for some particular use. All these points aks for soft-updates to be configured per mount-point, not per disk partition (i.e. *not* in the super-block). Regarding 3: I have a lot of systems in remote locations, some with only limited access to the console. (For example, there may be a modem that is switched off, unless activated for the duration of a remote access.) If I want to enable soft-updates on tens of machines, I'd rather only distribute a configuration file, than connect to each machine, boot into single-user mode, enable soft-updates on each file-system with tunefs, and then let the system go back to multi-user. There are other scenarios, where a system may receive a completely new kernel, without an administrator present, who knows how to use tunefs. (Think about PicoBSD: It should be possible to have soft-updates enabled or disabled by a configuration file delivered with the boot floppy.) The natural place for persistent per file-system configuration is either the super-block, where disklabel or newfs store immutable parameters, that can't be modified without loosing all data on a partition. The other place where such information is kept is /etc/fstab, which holds the usage, security and update policy decisions of a system in per records per mount-point. (Even if the first column in fstab is the partition name, I consider fstab to specify what to mount on a *mount point* and how to do it. but I admit that this is just my personal POV ;-) It is of course possible the have /etc/rc (or one of the rc.* sub-tasks) modify soft-updates before the "mount -a". But that would be more complex than my proposed solution. > Your above proposal would work, though that is not how NetBSD > implemented it. I feel that it is a lot of extra mechanism for very > little gain. Administrators generally make a one-time decision to > run soft updates on a filesystem. It is not the sort of thing that > they want to change on a regular basis. It is possible to run tunefs Agreed. But there are multiple places where this information may be stored. One is to hide it in the super-block, the other to expose it in /etc/fstab. Since mount reports the soft-updates state, I think the POLA asks for mount to be able to choose between softdep/nosoftdep. > on a filesystem that is mounted read-only, so it no more difficult > to use tunefs than it is to make it a mount-time option (i.e., they > still have to down-grade to read-only, set the option, then upgrade). This is not easily possible for some file-systems (like /var), where some processes keep files open for writing at all times and may fail, if the file system mode is forced to R/O. > Finally, I expect that soft updates will eventually just be defaulted > to `on' when a filesystem is built, and in a few rare instances an > administrator will want to turn it off. I do not want to have an > option that needs to be added to nearly every fstab entry to get > the default behavior. I fully agree that the default should not have to be added to each line in fstab. But then it could still default to on in the kernel, and a "nosoftdep" could disable it for selected file systems. But I see your point. Hmmm, I still do not think that the command character of FS_DOSOFTDEP and the reporting character (was previously mounted with soft-updates) of the same bit are a good design choice. The two functions should be separated. One way to do this is by making soft-updates a mount option as I propose, the other to have two flag bits in the super-block, one enable soft-updates, the other to report to fsck whether it was enabled. > Plus it is just one more bit of trivia that > new system administrators need to learn to make their systems run > well. The more of those details that need not be learned because > they just do the right thing, the better. But the tunefs method is anything but obvious. Well, if the next version of "sysinstall" offers to enable soft-updates in the partition editor, it will not be missed as easily as currently ... I think that soft-updates should be on by default, except possibly on the root file-system (which may happen to be too small: at least 15MB of free space are required for a "make installworld" or part of the programs in /bin and /sbin will be lost). There should of course not be different defaults for / and other file-systems, since this would violate the POLA. I'm still not convinced. I think that soft-updates should be enabled by default, and some people may want to disable it on the root fs ... I am writing this message on a system, which is modified to make softdep a mount option. This required a total of some 10 additional lines (incl. comments) to be added to ffs_vfsops.c, plus a few lines that changed from fs_flags to mnt_flag. A total of 3 lines had to be added to the mount program (two to enable/disable softdep, one to print its state for "mount -p"). Patches are appended below, just to show that this is a very minor change. Please consider adding these patches at least at the time, when soft-updates defaults to "on" ... Best regards, STefan ***************************************************************************** Patches to make softdep a "mount" option: -> Initial mount (no update mount, file system currently not mounted): The following patches make ffs_mount() perform a soft-updates mount (which sets the FS_DOSOFTDEP bit in fs->fs_flags, if successful), if all of the following conditions are true: 1) the MNT_SOFTDEP flag is set in mp->mnt_flag 2) soft-updates is compiled into the kernel 3) the file system is unmounted or mounted read-only The FS_DOSOFTDEP bit is cleared, if the mount succeeds without MNT_SOFTDEP being set. This can be caused by either of: 1) a successful mount without the "-o softdep" option (or the nosoftdep option, if softdep is to become the default). 2) a mount on a system without soft-updates support -> Update mount (typically: read-only -> read-write): If this is an update mount and the file-system is not clean, then the soft-updates enable status is "locked" (which is an even stronger condition than the requirement, that the file system be unmounted for "tunefs -n enable", which doesn't prevent changing an un-clean file system and mis-directing a subsequent fsck run ...). (If the file system is mounted R/O, the situation is identical to the "Initial mount" case above.) --- /sys/ufs/ffs/ffs_softdep.c 2000/06/22 00:29:53 1.69 +++ /sys/ufs/ffs/ffs_softdep.c 2000/06/27 18:22:49 @@ -962,5 +962,5 @@ mp->mnt_flag &= ~MNT_ASYNC; - mp->mnt_flag |= MNT_SOFTDEP; + fs->fs_flags |= FS_DOSOFTDEP; /* * When doing soft updates, the counters in the --- /sys/ufs/ffs/ffs_softdep_stub.c 2000/06/16 13:00:33 1.11 +++ /sys/ufs/ffs/ffs_softdep_stub.c 2000/06/27 18:15:44 @@ -72,4 +72,5 @@ struct ucred *cred; { + mp->mnt_flag &= ~MNT_SOFTDEP; return (0); --- /sys/ufs/ffs/ffs_vfsops.c 2000/06/04 04:50:36 1.122 +++ /sys/ufs/ffs/ffs_vfsops.c 2000/06/27 20:33:37 @@ -196,4 +196,12 @@ err = 0; ronly = fs->fs_ronly; /* MNT_RELOAD might change this */ + if (ronly == 0) { + /* preserve current softdep status, unless mounted read-only */ + if (fs->fs_flags & FS_DOSOFTDEP) { + mp->mnt_flag |= MNT_SOFTDEP; + } else { + mp->mnt_flag &= ~MNT_SOFTDEP; + } + } if (ronly == 0 && (mp->mnt_flag & MNT_RDONLY)) { flags = WRITECLOSE; @@ -244,5 +252,5 @@ /* check to see if we need to start softdep */ - if (fs->fs_flags & FS_DOSOFTDEP) { + if (mp->mnt_flag & MNT_SOFTDEP) { err = softdep_mount(devvp, mp, fs, p->p_ucred); if (err) @@ -658,4 +666,7 @@ goto out; } + /* clear softdep flag in superblock, if not a softdep mount */ + if (mp->mnt_flag & MNT_SOFTDEP == 0) + fs->fs_flags &= ~FS_DOSOFTDEP; ump = malloc(sizeof *ump, M_UFSMNT, M_WAITOK); bzero((caddr_t)ump, sizeof *ump); @@ -746,5 +757,5 @@ fs->fs_maxfilesize = maxfilesize; /* XXX */ if (ronly == 0) { - if ((fs->fs_flags & FS_DOSOFTDEP) && + if ((mp->mnt_flag & MNT_SOFTDEP) && (error = softdep_mount(devvp, mp, fs, cred)) != 0) { free(base, M_UFSMNT); --- /usr/src/sbin/mount/mntopts.h 1999/10/09 11:54:07 1.16 +++ /usr/src/sbin/mount/mntopts.h 2000/06/27 19:06:19 @@ -57,4 +57,5 @@ #define MOPT_NOCLUSTERW { "clusterw", 1, MNT_NOCLUSTERW, 0 } #define MOPT_SUIDDIR { "suiddir", 0, MNT_SUIDDIR, 0 } +#define MOPT_SOFTDEP { "softdep", 0, MNT_SOFTDEP, 0 } /* Control flags. */ --- /usr/src/sbin/mount/mount.c 2000/01/15 14:28:13 1.39 +++ /usr/src/sbin/mount/mount.c 2000/06/27 20:14:39 @@ -741,4 +741,5 @@ if (flags & MNT_NOSYMFOLLOW) res = catopt(res, "nosymfollow"); if (flags & MNT_SUIDDIR) res = catopt(res, "suiddir"); + if (flags & MNT_SOFTDEP) res = catopt(res, "softdep"); return res; --- /usr/src/sbin/mount/mount_ufs.c 1999/08/28 00:13:27 1.16 +++ /usr/src/sbin/mount/mount_ufs.c 2000/06/27 17:26:50 @@ -67,4 +67,5 @@ MOPT_SYNC, MOPT_UPDATE, + MOPT_SOFTDEP, { NULL } };