Date: Fri, 2 Dec 2011 02:08:55 -0800 From: Devin Teske <devin.teske@fisglobal.com> To: "'Doug Barton'" <dougb@FreeBSD.org> Cc: freebsd-rc@freebsd.org, 'Ken Smith' <kensmith@buffalo.edu>, 'Parker-Smith' <daveps@vicor.com>, phk@freebsd.org, 'Julian Elischer' <julian@freebsd.org>, Dave@FreeBSD.ORG Subject: RE: mount(8) bug? rc.d/mountlate bug? bug in both? Message-ID: <039f01ccb0da$67b50f20$371f2d60$@fisglobal.com> In-Reply-To: <4ED862D6.9090807@FreeBSD.org> References: <039201ccb0ab$b3db9470$1b92bd50$@fisglobal.com> <4ED862D6.9090807@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
> -----Original Message-----
> From: Doug Barton [mailto:dougb@FreeBSD.org]
> Sent: Thursday, December 01, 2011 9:32 PM
> To: Devin Teske
> Cc: freebsd-rc@freebsd.org; Ken Smith; Parker-Smith; Dave@FreeBSD.ORG;
> phk@freebsd.org; 'Julian Elischer'
> Subject: Re: mount(8) bug? rc.d/mountlate bug? bug in both?
>
> Short answer, tag the mount(s) noauto in fstab, and mount them in
/etc/rc.local.
>
That may be the simplest approach.
However, we're looking more for a solution that involves keeping the NFS mounts
in fstab(5).
Reason: In planning-to-upgrade hundreds of machines -- upgrading from
FreeBSD-4.11 (where fstab(5) NFS failures are non-fatal) to FreeBSD-8.1 (where
they are) -- the process would go smoother if we didn't have to migrate fstab(5)
NFS entries in the manner you describe above.
Researching and testing further, it appears that the "bg" option is almost
entirely what we want.
I was able to successfully achieve what I wanted with the following in fstab(5):
bogus:/bogus /bogus nfs tcp,rw,nosuid,bg,late 0 0
AND adding to /etc/hosts:
128.0.0.1 bogus
A timeout occurs attempting to connect to the system named "bogus" and upon
failure, mount_nfs(8) properly daemon(3)izes into the background, re-attempting
every 60 seconds to mount the volume.
However, if I take out the /etc/hosts entry shown above (making the name
unresolvable), mount_nfs(8) will this time exit immediately with error status
rather than adhering to the "bg" option (which, for all intents and purposes,
just because the netid for the host isn't accessible at the time, doesn't mean
it won't be in 60 seconds -- thus justifying the logic that the "bg" option
should apply even when the netid can't be obtained, if not especially-so).
I propose the following [UNTESTED] patch, which tries to make remote-errors
non-fatal to a filesystem marked as "bg".
========== BEGIN PATCH EXCERPT ==========
--- sbin/mount_nfs/mount_nfs.c.orig Fri Dec 2 00:46:44 2011
+++ sbin/mount_nfs/mount_nfs.c Fri Dec 2 01:20:38 2011
@@ -803,9 +803,17 @@ getnfsargs(char *spec, struct iovec **io
if (ret == TRYRET_SUCCESS)
break;
- /* Exit if all errors were local. */
- if (!remoteerr)
- exit(1);
+ if ((opflags & (BGRND | ISBGRND)) == BGRND) {
+ warnx("Cannot immediately mount %s:%s, backgrounding",
+ hostp, spec);
+ opflags |= ISBGRND;
+ if (daemon(0, 0) != 0)
+ err(1, "daemon");
+ } else {
+ /* Exit if all errors were local. */
+ if (!remoteerr)
+ exit(1);
+ }
/*
* If retrycnt == 0, we are to keep retrying forever.
@@ -814,13 +822,6 @@ getnfsargs(char *spec, struct iovec **io
if (retrycnt != 0 && --retrycnt == 0)
exit(1);
- if ((opflags & (BGRND | ISBGRND)) == BGRND) {
- warnx("Cannot immediately mount %s:%s, backgrounding",
- hostp, spec);
- opflags |= ISBGRND;
- if (daemon(0, 0) != 0)
- err(1, "daemon");
- }
sleep(60);
}
freeaddrinfo(ai_nfs);
========== END PATCH EXCERPT ==========
Other comments on responses to initial situation below.
> More detailed analysis of your situation follows.
>
> On 12/01/2011 20:34, Devin Teske wrote:
> > Hi -RC@, Julian, Poul, and Ken,
> >
> > We need your help on FreeBSD-8.1!
> >
> > Please read the following dossier on our issue with simply attempting
> > to add a single NFS mount to fstab(5) ***without*** the side-effect of
> > rebooting into single-user mode should that mount fail for ANY reason during
> boot.
> >
> > FULL-DISCLOSURE: We've already tried marking the filesystem as "late"
> > and/or "bg" to no avail. We've traced the problem down to a possible
> > bug in either
> > mount(8) or the `/etc/rc.d/mountlate' boot-script. Need confirmation
> > that this is a bug, OR a work-around to eliminate the numerous
> > edge-cases where we can reliably cause the system to boot into single-user
> mode.
>
> I don't think it's a bug in either.
You may be right. More below.
>
> > Corollary:
> >
> > Having a workstation 3000+ miles away in India reboot into single-user
> > mode simply because of a momentary network hiccup (or any other
> > situation that could cause failure of the NFS mount) at boot is what
> > we're trying to avoid. That is to explain, avoiding the situation
> > where a system that is physically afar from becoming permanently
> > unresponsive, requiring significant expenditure/effort to rectify.
>
> If you're administrating a remote system it's reasonable to assume that you
have
> a serial console on it. That said, we're certainly not *trying* to break stuff
willy-
> nilly.
No serial console enabled on the workstations/desktops.
>
> > Possible Bug:
> >
> > As the system is booting, /etc/rc.d/mountcritremote attempts to mount
> > the filesystem. It fails. This is OK (because mountcritremote does not
> > return FAILURE status -- he returns SUCCESS and boot proceeds as-expected).
> >
> > Later, /etc/rc.d/mountlate runs and attempts to mount it again. It
> > fails again except this time mountlate calls "stop_boot" after the
> > failure (dropping us to single-user mode).
>
> This the expected/desired behavior.
Fair enough.
>
> > The "possible bug" comes into play in reading /etc/rc.d/mountlate and
> > finding out just how exactly it determines that it should have been
> > mounting this filesystem in the first place.
> >
> > mountlate calls "/sbin/mount -d -a -l" to determine if there are any "late"
> > filesystems to mount.
>
> No. The -a in there means that it's looking for *all* unmounted file systems,
> including those marked "late."
Correct.
>
> In fact a key reason for the division between
> mountcritremote and mountlate is that circumstances on the system may have
> changed to allow things that failed the first time to be mounted, *in addition
to*
> specifically marking certain entries "late" because we know that they cannot
> succeed until later in the boot.
>
Right.
> > The filesystem is NOT marked as "late", but "/sbin/mount -d -a -l"
> > will still report it because it's not yet mounted.
>
> Right-O.
>
All good.
--
Devin
_____________
The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?039f01ccb0da$67b50f20$371f2d60$>