From owner-freebsd-bugs@FreeBSD.ORG Wed Jan 5 07:30:31 2005 Return-Path: Delivered-To: freebsd-bugs@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 930F116A4CE for ; Wed, 5 Jan 2005 07:30:31 +0000 (GMT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2517C43D48 for ; Wed, 5 Jan 2005 07:30:31 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.1/8.13.1) with ESMTP id j057UVXb093467 for ; Wed, 5 Jan 2005 07:30:31 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.13.1/8.13.1/Submit) id j057UUVK093466; Wed, 5 Jan 2005 07:30:31 GMT (envelope-from gnats) Resent-Date: Wed, 5 Jan 2005 07:30:31 GMT Resent-Message-Id: <200501050730.j057UUVK093466@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-bugs@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Dirk-Willem van Gulik Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6BFA416A4CE for ; Wed, 5 Jan 2005 07:23:24 +0000 (GMT) Received: from www.freebsd.org (www.freebsd.org [216.136.204.117]) by mx1.FreeBSD.org (Postfix) with ESMTP id 11F1943D2D for ; Wed, 5 Jan 2005 07:23:24 +0000 (GMT) (envelope-from nobody@FreeBSD.org) Received: from www.freebsd.org (localhost [127.0.0.1]) by www.freebsd.org (8.13.1/8.13.1) with ESMTP id j057NLqT054337 for ; Wed, 5 Jan 2005 07:23:21 GMT (envelope-from nobody@www.freebsd.org) Received: (from nobody@localhost) by www.freebsd.org (8.13.1/8.13.1/Submit) id j057NLN8054336; Wed, 5 Jan 2005 07:23:21 GMT (envelope-from nobody) Message-Id: <200501050723.j057NLN8054336@www.freebsd.org> Date: Wed, 5 Jan 2005 07:23:21 GMT From: Dirk-Willem van Gulik To: freebsd-gnats-submit@FreeBSD.org X-Send-Pr-Version: www-2.3 Subject: misc/75842: valid, but double, fstab mount-point entries ignored by mount -a at boot time. X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2005 07:30:31 -0000 >Number: 75842 >Category: misc >Synopsis: valid, but double, fstab mount-point entries ignored by mount -a at boot time. >Confidential: no >Severity: non-critical >Priority: low >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Wed Jan 05 07:30:30 GMT 2005 >Closed-Date: >Last-Modified: >Originator: Dirk-Willem van Gulik >Release: 5.3 >Organization: Stichting Wireless Leiden >Environment: FreeBSD kobus.cnodelcpl.wleiden.net 5.3-STABLE FreeBSD 5.3-STABLE #2: Wed Jan 5 00:45:11 CET 2005 root@kobus.cnodelcpl.wleiden.net:/usr/obj/usr/src/sys/KOBUS i386 >Description: mount -a ignores the second (or third) entry for a given mountpoint at boot time. Which seems sensible - but I found the following situation where that is in fact needed: When using jails it is often desirable to give each jail a virtual playground - rather than a real '/' as to safe replicating a '/' for every jail. This can be accomplished by using a unionfs over a read-only root and overlaying it with a small partition which can be read/write. This partition can also be a disk image or a swap/memory backed image. However this means listing the partition twice in the /etc/fstab file. And for some reason a mount at boot time ignores this. >How-To-Repeat: Create a file backed disk with mkdir -p /usr/tmp dd if=/dev/zero of=/usr/tmp/md.fs bs=1k count=5k N=`mdconfig -a -t vnode -f /usr/tmp/md.fs` bsdlabel -w /dev/$N auto newfs /dev/$N\c mount /dev/$N\c /mnt create a read only version of some random partition/directory mkdir /sys-ro mount_nullfs -o ro /sys /sys-ro and mount this cross the writable virtual disk mount_unionfs -r /sys-ro /mnt Check that it works. Unount it all umount /mnt umount /sys-ro umount /mnt # remember - need to do this TWICE and add to /etc/fstab (mount -p should work but it does not - see PR 75585) the following: md /mnt mfs rw,-F/usr/tmp/md.fs 0 0 /sys /sys-ro nullfs ro 0 0 /sys-ro /mnt unionfs rw,noclusterw,-b 0 0 then do 'mount -a' to check and then REBOOT to observe it missing the second /mnt entry. Manual mounts are still fine. A more complete/realistic/useful test is below. Create a set of file backed disks with: DIR=/usr/mdfs mkdir -p $DIR for i in 130 131 132 333 134 135 136 137 138 139 140 141 142 do dd if=/dev/zero of=$DIR/mdimage.$i.fs bs=1k count=5k D=`mdconfig -a -t vnode -f $DIR/mdimage.$i.fs` bsdlabel -w /dev/$D auto newfs /dev/$D\c done Then add the following to fstab to get these mounted at boot time. md /usr/172.21.16.130 mfs rw,-F/usr/mdfs/mdimage.130.fs 0 0 md /usr/172.21.16.131 mfs rw,-F/usr/mdfs/mdimage.131.fs 0 0 md /usr/172.21.16.132 mfs rw,-F/usr/mdfs/mdimage.132.fs 0 0 md /usr/172.21.16.133 mfs rw,-F/usr/mdfs/mdimage.133.fs 0 0 md /usr/172.21.16.134 mfs rw,-F/usr/mdfs/mdimage.134.fs 0 0 md /usr/172.21.16.135 mfs rw,-F/usr/mdfs/mdimage.135.fs 0 0 md /usr/172.21.16.136 mfs rw,-F/usr/mdfs/mdimage.136.fs 0 0 md /usr/172.21.16.137 mfs rw,-F/usr/mdfs/mdimage.137.fs 0 0 md /usr/172.21.16.138 mfs rw,-F/usr/mdfs/mdimage.138.fs 0 0 md /usr/172.21.16.139 mfs rw,-F/usr/mdfs/mdimage.139.fs 0 0 md /usr/172.21.16.140 mfs rw,-F/usr/mdfs/mdimage.140.fs 0 0 md /usr/172.21.16.141 mfs rw,-F/usr/mdfs/mdimage.141.fs 0 0 md /usr/172.21.16.142 mfs rw,-F/usr/mdfs/mdimage.142.fs 0 0 Next create a jail-base - and overlay it so that it can be mounted read-only later. add the following to ./etc/fstab: /usr/jail-base /usr/jail-base-ro nullfs ro 0 0 Next add the following to fstab to make an overlay for the read only jail /usr/jail-base-ro /usr/172.21.16.130 unionfs rw,noclusterw,-b 0 0 /usr/jail-base-ro /usr/172.21.16.131 unionfs rw,noclusterw,-b 0 0 /usr/jail-base-ro /usr/172.21.16.132 unionfs rw,noclusterw,-b 0 0 /usr/jail-base-ro /usr/172.21.16.133 unionfs rw,noclusterw,-b 0 0 /usr/jail-base-ro /usr/172.21.16.134 unionfs rw,noclusterw,-b 0 0 /usr/jail-base-ro /usr/172.21.16.135 unionfs rw,noclusterw,-b 0 0 /usr/jail-base-ro /usr/172.21.16.136 unionfs rw,noclusterw,-b 0 0 /usr/jail-base-ro /usr/172.21.16.137 unionfs rw,noclusterw,-b 0 0 /usr/jail-base-ro /usr/172.21.16.138 unionfs rw,noclusterw,-b 0 0 /usr/jail-base-ro /usr/172.21.16.139 unionfs rw,noclusterw,-b 0 0 /usr/jail-base-ro /usr/172.21.16.140 unionfs rw,noclusterw,-b 0 0 /usr/jail-base-ro /usr/172.21.16.141 unionfs rw,noclusterw,-b 0 0 /usr/jail-base-ro /usr/172.21.16.142 unionfs rw,noclusterw,-b 0 0 And then add the following to your rc.conf jail_enable="YES" jail_set_hostname_allow="YES" jail_socket_unixiproute_only="YES" jail_sysvipc_allow="NO" jail_stop_jailer="NO" jail_list="130 131 132 133 134 135 136 137 138 139 140 141 142" jail_130_exec="/usr/local/sbin/jailer" jail_130_devfs_enable="YES" # mount devfs in the jail jail_130_descfs_enable="YES" # mount fdescfs in the jail jail_130_procfs_enable="YES" # mount procfs in jail jail_130_devfs_ruleset="130" # devfs ruleset to apply to jail jail_130_rootdir="/usr/172.21.16.130" # Jail's root directory jail_130_hostname="j-130.wleiden.net" # Jail's hostname jail_130_ip="172.21.16.130" # Jail's IP number jail_131_exec="/usr/local/sbin/jailer" jail_131_devfs_enable="YES" # mount devfs in the jail jail_131_descfs_enable="YES" # mount fdescfs in the jail jail_131_procfs_enable="YES" # mount procfs in jail jail_131_devfs_ruleset="131" # devfs ruleset to apply to jail jail_131_rootdir="/usr/172.21.16.131" # Jail's root directory jail_131_hostname="j-131.wleiden.net" # Jail's hostname jail_131_ip="172.21.16.131" # Jail's IP number jail_132_exec="/usr/local/sbin/jailer" jail_132_devfs_enable="YES" # mount devfs in the jail jail_132_descfs_enable="YES" # mount fdescfs in the jail jail_132_procfs_enable="YES" # mount procfs in jail jail_132_devfs_ruleset="132" # devfs ruleset to apply to jail jail_132_rootdir="/usr/172.21.16.132" # Jail's root directory jail_132_hostname="j-132.wleiden.net" # Jail's hostname jail_132_ip="172.21.16.132" # Jail's IP number jail_133_exec="/usr/local/sbin/jailer" jail_133_devfs_enable="YES" # mount devfs in the jail jail_133_descfs_enable="YES" # mount fdescfs in the jail jail_133_procfs_enable="YES" # mount procfs in jail jail_133_devfs_ruleset="133" # devfs ruleset to apply to jail jail_133_rootdir="/usr/172.21.16.133" # Jail's root directory jail_133_hostname="j-133.wleiden.net" # Jail's hostname jail_133_ip="172.21.16.133" # Jail's IP number jail_134_exec="/usr/local/sbin/jailer" jail_134_devfs_enable="YES" # mount devfs in the jail jail_134_descfs_enable="YES" # mount fdescfs in the jail jail_134_procfs_enable="YES" # mount procfs in jail jail_134_devfs_ruleset="134" # devfs ruleset to apply to jail jail_134_rootdir="/usr/172.21.16.134" # Jail's root directory jail_134_hostname="j-134.wleiden.net" # Jail's hostname jail_134_ip="172.21.16.134" # Jail's IP number jail_135_exec="/usr/local/sbin/jailer" jail_135_devfs_enable="YES" # mount devfs in the jail jail_135_descfs_enable="YES" # mount fdescfs in the jail jail_135_procfs_enable="YES" # mount procfs in jail jail_135_devfs_ruleset="135" # devfs ruleset to apply to jail jail_135_rootdir="/usr/172.21.16.135" # Jail's root directory jail_135_hostname="j-135.wleiden.net" # Jail's hostname jail_135_ip="172.21.16.135" # Jail's IP number jail_136_exec="/usr/local/sbin/jailer" jail_136_devfs_enable="YES" # mount devfs in the jail jail_136_descfs_enable="YES" # mount fdescfs in the jail jail_136_procfs_enable="YES" # mount procfs in jail jail_136_devfs_ruleset="136" # devfs ruleset to apply to jail jail_136_rootdir="/usr/172.21.16.136" # Jail's root directory jail_136_hostname="j-136.wleiden.net" # Jail's hostname jail_136_ip="172.21.16.136" # Jail's IP number jail_137_exec="/usr/local/sbin/jailer" jail_137_devfs_enable="YES" # mount devfs in the jail jail_137_descfs_enable="YES" # mount fdescfs in the jail jail_137_procfs_enable="YES" # mount procfs in jail jail_137_devfs_ruleset="137" # devfs ruleset to apply to jail jail_137_rootdir="/usr/172.21.16.137" # Jail's root directory jail_137_hostname="j-137.wleiden.net" # Jail's hostname jail_137_ip="172.21.16.137" # Jail's IP number jail_138_exec="/usr/local/sbin/jailer" jail_138_devfs_enable="YES" # mount devfs in the jail jail_138_descfs_enable="YES" # mount fdescfs in the jail jail_138_procfs_enable="YES" # mount procfs in jail jail_138_devfs_ruleset="138" # devfs ruleset to apply to jail jail_138_rootdir="/usr/172.21.16.138" # Jail's root directory jail_138_hostname="j-138.wleiden.net" # Jail's hostname jail_138_ip="172.21.16.138" # Jail's IP number jail_139_exec="/usr/local/sbin/jailer" jail_139_devfs_enable="YES" # mount devfs in the jail jail_139_descfs_enable="YES" # mount fdescfs in the jail jail_139_procfs_enable="YES" # mount procfs in jail jail_139_devfs_ruleset="139" # devfs ruleset to apply to jail jail_139_rootdir="/usr/172.21.16.139" # Jail's root directory jail_139_hostname="j-139.wleiden.net" # Jail's hostname jail_139_ip="172.21.16.139" # Jail's IP number jail_140_exec="/usr/local/sbin/jailer" jail_140_devfs_enable="YES" # mount devfs in the jail jail_140_descfs_enable="YES" # mount fdescfs in the jail jail_140_procfs_enable="YES" # mount procfs in jail jail_140_devfs_ruleset="140" # devfs ruleset to apply to jail jail_140_rootdir="/usr/172.21.16.140" # Jail's root directory jail_140_hostname="j-140.wleiden.net" # Jail's hostname jail_140_ip="172.21.16.140" # Jail's IP number jail_141_exec="/usr/local/sbin/jailer" jail_141_devfs_enable="YES" # mount devfs in the jail jail_141_descfs_enable="YES" # mount fdescfs in the jail jail_141_procfs_enable="YES" # mount procfs in jail jail_141_devfs_ruleset="141" # devfs ruleset to apply to jail jail_141_rootdir="/usr/172.21.16.141" # Jail's root directory jail_141_hostname="j-141.wleiden.net" # Jail's hostname jail_141_ip="172.21.16.141" # Jail's IP number jail_142_exec="/usr/local/sbin/jailer" jail_142_devfs_enable="YES" # mount devfs in the jail jail_142_descfs_enable="YES" # mount fdescfs in the jail jail_142_procfs_enable="YES" # mount procfs in jail jail_142_devfs_ruleset="142" # devfs ruleset to apply to jail jail_142_rootdir="/usr/172.21.16.142" # Jail's root directory jail_142_hostname="j-142.wleiden.net" # Jail's hostname jail_142_ip="172.21.16.142" # Jail's IP number >Fix: mount manually or 'fool' -a by using a symlink. In the above example: ln -s /mnt /mnt-fake and then change the last fstab entry into /sys-ro /mnt-fake unionfs rw,nocluserw,-b 0 0 Or for the full example do the same and then change: /usr/jail-base-ro /usr/172.21.16.130-m unionfs rw,noclusterw,-b 0 0 /usr/jail-base-ro /usr/172.21.16.131-m unionfs rw,noclusterw,-b 0 0 /usr/jail-base-ro /usr/172.21.16.132-m unionfs rw,noclusterw,-b 0 0 /usr/jail-base-ro /usr/172.21.16.133-m unionfs rw,noclusterw,-b 0 0 /usr/jail-base-ro /usr/172.21.16.134-m unionfs rw,noclusterw,-b 0 0 /usr/jail-base-ro /usr/172.21.16.135-m unionfs rw,noclusterw,-b 0 0 /usr/jail-base-ro /usr/172.21.16.136-m unionfs rw,noclusterw,-b 0 0 /usr/jail-base-ro /usr/172.21.16.137-m unionfs rw,noclusterw,-b 0 0 /usr/jail-base-ro /usr/172.21.16.138-m unionfs rw,noclusterw,-b 0 0 /usr/jail-base-ro /usr/172.21.16.139-m unionfs rw,noclusterw,-b 0 0 /usr/jail-base-ro /usr/172.21.16.140-m unionfs rw,noclusterw,-b 0 0 /usr/jail-base-ro /usr/172.21.16.141-m unionfs rw,noclusterw,-b 0 0 /usr/jail-base-ro /usr/172.21.16.142-m unionfs rw,noclusterw,-b 0 0 >Release-Note: >Audit-Trail: >Unformatted: