From owner-freebsd-bugs@FreeBSD.ORG Fri Dec 2 17:50:03 2005 Return-Path: X-Original-To: freebsd-bugs@hub.freebsd.org Delivered-To: freebsd-bugs@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3CFE116A41F for ; Fri, 2 Dec 2005 17:50:03 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4451043D6B for ; Fri, 2 Dec 2005 17:50:02 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.3/8.13.3) with ESMTP id jB2Ho2Nd059863 for ; Fri, 2 Dec 2005 17:50:02 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.13.3/8.13.1/Submit) id jB2Ho26X059862; Fri, 2 Dec 2005 17:50:02 GMT (envelope-from gnats) Resent-Date: Fri, 2 Dec 2005 17:50:02 GMT Resent-Message-Id: <200512021750.jB2Ho26X059862@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-bugs@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Anders Nordby Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3999F16A41F; Fri, 2 Dec 2005 17:41:26 +0000 (GMT) (envelope-from anders@totem.fix.no) Received: from totem.fix.no (totem.fix.no [80.91.36.20]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9DA8E43D53; Fri, 2 Dec 2005 17:41:25 +0000 (GMT) (envelope-from anders@totem.fix.no) Received: by totem.fix.no (Postfix, from userid 1000) id 0B3E88DB10D; Fri, 2 Dec 2005 18:41:23 +0100 (CET) Message-Id: <20051202174123.0B3E88DB10D@totem.fix.no> Date: Fri, 2 Dec 2005 18:41:23 +0100 (CET) From: Anders Nordby To: FreeBSD-gnats-submit@FreeBSD.org X-Send-Pr-Version: 3.113 Cc: rse@FreeBSD.org, pjd@FreeBSD.org, phk@FreeBSD.org Subject: conf/89860: Add ability to set up vnode backed filesystems for jails in rc.conf X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Anders Nordby List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Dec 2005 17:50:03 -0000 >Number: 89860 >Category: conf >Synopsis: Add ability to set up vnode backed filesystems for jails in rc.conf >Confidential: no >Severity: non-critical >Priority: low >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: change-request >Submitter-Id: current-users >Arrival-Date: Fri Dec 02 17:50:01 GMT 2005 >Closed-Date: >Last-Modified: >Originator: Anders Nordby >Release: FreeBSD 6.0-RELEASE i386 >Organization: Fupp >Environment: System: FreeBSD totem.fix.no 6.0-RELEASE FreeBSD 6.0-RELEASE #0: Mon Nov 21 01:52:03 UTC 2005 root@master.fupp.net:/usr/obj/usr/src/sys/MASTER i386 >Description: Currently, if you want your jail's root filesystems mounted off md(4) vnode backed filesystems, you need to configure them first. This is a bit awkward, cause you need to have custom scripts and/or combine with jail setup in /etc/rc.conf. (It is a benefit to mount jail filesystems off a vnode backed md(4) device to keep its diskspace usage limited, so that it does not fill up diskspace for other jails.) >How-To-Repeat: >Fix: I added functionality for this in the jail rc-script. This is my first cut. Maybe something needs to be improved. Any feedback is most welcome. Global options: jail_vnmount_fsck="YES" jail_vnmount_fsck_y_enable="YES" jail_vnmount_mountopts="-o noatime" #jail_vnmount_mdconfigopts="-o async" Per jail options: jail_foo_vnmount="/data/jail/foo-fs:c=/data/jail/foo /data/jail/foo-fs2=/data/jail/foo/foobar" jail_bar_vnmount="/data/jail/bar-fs:s1a=/data/jail/bar" Adding jail_ will attempt to configure, fsck and mount the vnode filesystems, specified as pairs of :=. The : is optional, if empty c (the whole memory disk) is used. It could be beneficial to configure the md(4) devices using -o async (cause otherwise performance sucks), but mounting with -o noatime helps a bit. Hopefully md(4) and async mode will be fixed and free from deadlocks one day. :-) I added my touches to the sc script after rse's unmount, and before mount so that the vnmount and mount options can both be used for the same jail. My patch is made for and with FreeBSD 6.0-RELEASE, but applies fine to the HEAD/-current version also. There is only one practical problem that I see with it now. If you try to stop a jail with interactive running processes, even though all processes die and filesystems are umounted fine, the jail id does not disappar. By starting and stopping a lot, you get duplicates. It doesn't prevent the same jail from being started and stopped again however. Here we go: --- rc.d/jail.old Thu Dec 1 15:00:52 2005 +++ rc.d/jail Fri Dec 2 18:28:49 2005 @@ -59,6 +59,10 @@ eval jail_procfs=\"\$jail_${_j}_procfs_enable\" [ -z "${jail_procfs}" ] && jail_procfs="NO" + eval jail_vnmount=\"\$jail_${_j}_vnmount\" + [ -z "${jail_vnmount_fsck}" ] && jail_vnmount_fsck="YES" + [ -z "${jail_vnmount_fsck_y_enable}" ] && jail_vnmount_fsck_y_enable="NO" + [ -z "${jail_mount}" ] && jail_mount="YES" eval jail_mount=\"\$jail_${_j}_mount_enable\" [ -z "${jail_mount}" ] && jail_mount="NO" # "/etc/fstab.${_j}" will be used for {,u}mount(8) if none is specified. @@ -72,6 +76,11 @@ debug "$_j devfs enable: $jail_devfs" debug "$_j fdescfs enable: $jail_fdescfs" debug "$_j procfs enable: $jail_procfs" + debug "$_j vnmount: $jail_vnmount" + debug "$_j vnmount_mdconfigopts: $jail_vnmount_mdconfigopts" + debug "$_j vnmount_mountopts: $jail_vnmount_mountopts" + debug "$_j vnmount_fsck: $jail_vnmount_fsck" + debug "$_j vnmount_fsck_y_enable: $jail_vnmount_fsck_y_enable" debug "$_j mount enable: $jail_mount" debug "$_j hostname: $jail_hostname" debug "$_j ip: $jail_ip" @@ -139,6 +148,19 @@ [ -f "${jail_fstab}" ] || warn "${jail_fstab} does not exist" umount -a -F "${jail_fstab}" >/dev/null 2>&1 fi + if [ -n "${jail_vnmount}" ]; then + for _mount_set in ${jail_vnmount}; do + _mountdir=${_mount_set#*=} + # Get md device name, only if it is an md device: + _dev=`mount | awk '{print $1, $3}' | egrep -E "^/dev/md[0-9a-z]+ ${_mountdir}$" | awk '{print $1}' | sed -E "s|^/dev/(md[0-9]+).*|\1|"` + if [ -n "$_dev" ]; then + if (umount -f $_mountdir >/dev/null 2>&1); then + mdconfig -d -u $_dev >/dev/null 2>&1 + fi + fi + done + + fi } jail_start() @@ -161,6 +183,36 @@ if [ -f /var/run/jail_${_jail}.id ]; then echo -n " [${jail_hostname} already running (/var/run/jail_${_jail}.id exists)]" continue; + fi + if [ -n "${jail_vnmount}" ]; then + for _mount_set in ${jail_vnmount}; do + _mountpair=${_mount_set%=*} + _vnfile=${_mountpair%:*} + _part=${_mountpair#*:} + _mountdir=${_mount_set#*=} + _dev=`mdconfig -a -t vnode ${jail_vnmount_mdconfigopts} -f $_vnfile 2>/dev/null` + if [ -n "$_dev" ]; then + if [ -z "$_part" -o "$_part" = "$_vnfile" ] + then + _part=c + fi + _mountdev="/dev/$_dev$_part" + + if checkyesno jail_vnmount_fsck; then + if checkyesno jail_vnmount_fsck_y_enable; then + fsck -y $_mountdev >/dev/null 2>&1 + else + fsck $_mountdev >/dev/null 2>&1 + fi + if [ "$?" -eq 0 ] + then + mount ${jail_vnmount_mountopts} $_mountdev $_mountdir >/dev/null 2>&1 + else + warn "could not fsck ${_mountdev}" + fi + fi + fi + done fi if checkyesno jail_mount; then info "Mounting fstab for jail ${_jail} (${jail_fstab})" --- defaults/rc.conf.orig Fri Dec 2 18:15:09 2005 +++ defaults/rc.conf Fri Dec 2 18:21:41 2005 @@ -518,6 +518,11 @@ jail_set_hostname_allow="YES" # Allow root user in a jail to change its hostname jail_socket_unixiproute_only="YES" # Route only TCP/IP within a jail jail_sysvipc_allow="NO" # Allow SystemV IPC use from within a jail +jail_vnmount_fsck="YES" # Fsck vnode backed jail filesystems +jail_vnmount_fsck_y_enable="YES" # Fsck vnode backed jail filesystems using -y +#jail_vnmount_mountopts="-o noatime" # Mount options for vnode backed jail filesystems +#jail_vnmount_mdconfigopts="-o async" # Mdconfig options for vnode backed jail filesystems + # # To use rc's built-in jail infrastructure create entries for @@ -536,6 +541,7 @@ #jail_example_devfs_ruleset="ruleset_name" # devfs ruleset to apply to jail #jail_example_fstab="" # fstab(5) for mount/umount #jail_example_flags="-l -U root" # flags for jail(8) +#jail_example_vnmount="/data/jail/example-fs:c=/data/jail/example # pairs of md/vnode filesystems to be mounted, :c part is partition and is optional ############################################################## ### Define source_rc_confs, the mechanism used by /etc/rc.* ## >Release-Note: >Audit-Trail: >Unformatted: