Date: Fri, 8 Jul 2016 13:28:53 -0500 From: Thomas Johnson <tommyj27@gmail.com> To: freebsd-jail@freebsd.org Subject: NFS + nullfs + jail = zombies? Message-ID: <CAMwYC7ZWH0s3h0AV8KxSLQk_NipvOkaMzkf2jF2Ty9Rx5x1H3w@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
I am working on developing a clustered application utilizing jails and
running into problems that seem to be NFS-related. I'm hoping that
someone can point out my error.
The jail images and my application data are served via NFS. The host
mounts NFS at boot, and then uses nullfs mounts to assemble the jail
tree when the jail is created (fstab files and jail.conf are below).
This seems to work fine, the jail starts and is usable. The problem
comes when I remove/restart the jail. Frequently (but not
consistently), the jail gets stuck in a dying state, causing the
unmount of the jail root (nullfs) to fail with a "device busy" error.
# jail -f /var/local/jail.conf -r wds1-1a
Stopping cron.
Waiting for PIDS: 1361.
.
Terminated
wds1-1a: removed
umount: unmount of /var/jail/wds1-1a failed: Device busy
# jls -av
JID Hostname Path
Name State
CPUSetID
IP Address(es)
1 wds1-1a /var/jail/wds1-1a
wds1-1a DYING
2
2620:1:1:1:1a::1
Through trial-and-error I have determined that forcing an unmount of
the root works, but subsequent mounts to that mount point will fail to
unmount with the same error. Deleting and recreating the mountpoint
fixes the mounting issue, but the dying jail remains permanently.
I have also found that if I copy the jail root to local storage and
update the jail's fstab to nullfs mount this, the problem seems to go
away. This leads me to believe that the issue is related to the NFS
source for the nullfs mount. statd and lockd are both running on the
host.
My relevant configurations are below. I can provide any other
information desired.
# Host fstab line for jail root.
#
10.219.212.1:/vol/dev/wds/jail_base /jail/base nfs ro 0 0
# Jail fstab file (mount.fstab)
#
/jail/base /var/jail/wds1-1a nullfs ro 0 0
# writable (UFS-backed) /var
/var/jail-vars/wds1-1a /var/jail/wds1-1a/var nullfs rw 0 0
# jail.conf file
#
* {
devfs_ruleset = "4";
mount.devfs;
exec.start = "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown";
interface = "vmx1";
allow.dying = 1;
exec.prestart = "/usr/local/bin/rsync -avC --delete
/jail/${image}/var/ /var/jail-vars/${host.hostname}/";
}
# JMANAGE wds1-1a
wds1-1a {
path = "/var/jail/wds1-1a";
ip6.addr = "2620:1:1:1:1a::1";
host.hostname = "wds1-1a";
host.domainname = "dev";
mount.fstab = "/var/local/fstab.wds1-1a";
$image = "base";
}
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAMwYC7ZWH0s3h0AV8KxSLQk_NipvOkaMzkf2jF2Ty9Rx5x1H3w>