From owner-freebsd-current@freebsd.org Thu Aug 29 12:32:10 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B75CED4F3D for ; Thu, 29 Aug 2019 12:32:10 +0000 (UTC) (envelope-from ohartmann@walstatt.org) Received: from mout.gmx.net (mout.gmx.net [212.227.15.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass Class 2 CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46K26x4h0Nz4c00 for ; Thu, 29 Aug 2019 12:32:09 +0000 (UTC) (envelope-from ohartmann@walstatt.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1567081922; bh=6NhPUtbGJ98g+iILBxHvvT02+lf5BQm1NSICMsP9cQY=; h=X-UI-Sender-Class:Date:From:To:Cc:Subject:In-Reply-To:References; b=hs0csv4V5/3eLsbzrd/HQcdfTQVRu3tYOTVRPuBmyuUZjV48zy/VsBvwVUysIXyza az04zl38ETAoJ52DidYwEM/S5MqtIELOvG2IM4lmZAC3vwv8ziiNJuf4sq22NIsPI/ U1myJMeDTPU2AjsoXbEATiXQxUaRJEkAoY8i6twg= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from freyja ([46.88.81.15]) by mail.gmx.com (mrgmx003 [212.227.17.190]) with ESMTPSA (Nemesis) id 0MBFUT-1htAdx3aRZ-00AEsE; Thu, 29 Aug 2019 14:26:49 +0200 Date: Thu, 29 Aug 2019 14:26:38 +0200 From: "O. Hartmann" To: Alexander Leidinger Cc: freebsd-current@freebsd.org, Michael Zhilin Subject: Re: jails, ZFS, deprecated jail variables and poudriere problems Message-ID: <20190829142632.6b93892b@freyja> In-Reply-To: <20190828135700.Horde.m2EjpS9j67KYn2E1oSeoK8f@webmail.leidinger.net> References: <20190827101149.1efcb946@freyja> <20190828135700.Horde.m2EjpS9j67KYn2E1oSeoK8f@webmail.leidinger.net> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable X-Provags-ID: V03:K1:4rlocoKUBzh0L+IhuuA9caHkIM5TifLBsUfPAbzojNMYtKOvpe2 oRsI1fwlCdq8H+GULDnyWKrd+TBy3IiREmP20pkvApl+9ZLvP/2NJe/qcWjfW74AGgngYkJ /uFxcowY/oyqX5N1fT+6j7lKQ3rdLhEau4NXAUE4Cy929HvqZ/s/Hjq7B/bN9ak1LiV1sP7 q52uKfp11ylI97nSr/Z8w== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:+K8F665sQ6I=:Q0vBspuatZg2X020Mjvntx 9Mlro2/zTkrv946uRX4dXaayHRTOxv5JQciWV+5z6Bk5Z5XnHJxq2xca74wzJIvf7PSsphXxt Ls91FArTqBiboQ+GKKFTF6PjClJzIwGUnAS9ljWvpyBjyiOS7eGNeK1pgCduBNP7L0BgZhchI 1rjOis6qPzuQesSacc56vKYnLaawlMx/+P0r8tvlsrQ/Ch3ons3UOpsYMC7Os8bDNXUkt/1/e fRGP/Klm3SQQxxRTMMdV/NOzbb94lW+dKOjIf//Rl/Lfa0fHuUX09qLGE+Z/8oNklYW+1E7EC GHZ94Ct+45YlHzjbC2RLpuIm0LnQEdHjMa+fzn0q/3ojEzg/87HHrdxGySbZ3B1d6iVLNDwrO EW/u+fEnR8UGhJRfCJSs0r6ouMiqMxV1uXWVUd8Wn2rXemwsnehia2hoFUeoIhW0sP3VDfwX/ J/Ta0psxbOqSX2K58MVjVmCd2+k65u+7fofk+q43/Vi7daQv1bB7NzwREiWMPBteUYIDJwBFt 9sNThchgNjlT9wQe2TNg3Hkjai9OYN+2lAZeqdLyL+fX8C/eywHXjeNXA+dLHs+Le/0wbmYx/ HMATAUJGztBhrp4PmpBJstSbq9i+QqJt5h2Ubh4uv7oeY0vSs6vYg2FQFauSxD5Xmk06gMh84 duD+mFi2mE4ubYUCYnr7yNV2nEvu+XqyOW/eT7DNqHWsTDyzYGNmY0fdr/tdUddxHy+zOSZit snaHRfxHsfaPySsEGl4KxUcEV2wUChhS6bdnr+ZhCi9E9H/awSDzmK1DswfOywfU1lPoIzDps ntegVMxF0us8DoqJZMG6Bpey96KrHyWl0Fhy46YKVo2UNmCrnLyfU4aHebfDRk9hPr+O/rV55 sxCgdwwIVxyGzoJKwx8vgHGhER+7w2tTFIfoHWgZ7Fob6uqyNvTDM+P0CqJdvDwzFgk5FBa6d 8JHUy/pP1q8vebiizVuLJ64MycFRM8yhcj4NP0XljCJ7yJtLeDrAtP7POreusv6fFvXOk98NU LF0HH9e3CEmzJX6jmvP1S6l489uyvUFd3o8OI8eN6mdO2wLiBDEOf1vvv+PFnK62Bqoso9Zi6 6JtMIPQXNai/QdQvTvRwyRnwt5jtbrIOzcXB1q4Z5COOLchd/snUz9ZFfFG+KXVKtI7nR50hM msiiFBrEUyxCt4nuECtBsnVwYHstbTAxTadM+tZvkQzjytPA== X-Rspamd-Queue-Id: 46K26x4h0Nz4c00 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmx.net header.s=badeba3b8450 header.b=hs0csv4V; dmarc=none; spf=none (mx1.freebsd.org: domain of ohartmann@walstatt.org has no SPF policy when checking 212.227.15.15) smtp.mailfrom=ohartmann@walstatt.org X-Spamd-Result: default: False [-3.98 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[gmx.net:s=badeba3b8450]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; IP_SCORE(-1.21)[ip: (-6.84), ipnet: 212.227.0.0/16(-1.38), asn: 8560(2.19), country: DE(-0.01)]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[walstatt.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[gmx.net:+]; NEURAL_HAM_SHORT(-0.97)[-0.969,0]; RCVD_IN_DNSWL_NONE(0.00)[15.15.227.212.list.dnswl.org : 127.0.3.0]; R_SPF_NA(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[15.81.88.46.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.10]; FROM_EQ_ENVFROM(0.00)[]; MID_RHS_NOT_FQDN(0.50)[]; ASN(0.00)[asn:8560, ipnet:212.227.0.0/16, country:DE]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Aug 2019 12:32:10 -0000 On Wed, 28 Aug 2019 13:57:00 +0200 Alexander Leidinger wrote: > Quoting "O. Hartmann" (from Tue, 27 Aug 2019 > 10:11:54 +0200): > > > We have a single ZFS pool (raidz), call it pool00 and this pool00 cona= tins a > > ZFS dataset pool00/poudriere which we want to exclusively attach to a = jail. > > pool00/poudriere contains a complete clone of a former, now decomissio= ned > > machine and is usable by the host bearing the jails. The jail, named > > poudriere, > > has these config parameters set in /etc/jail.conf as recommended: > > > > enforce_statfs=3D "0"; now set to enforce_statfs=3D "1"; > > > > allow.raw_sockets=3D "1"; > > > > allow.mount=3D "1"; > > allow.mount.zfs=3D "1"; > > The line above is what is needed, and what is replacing the sysctl > you've found. > > > allow.mount.devfs=3D "1"; > > allow.mount.fdescfs=3D "1"; > > allow.mount.procfs=3D "1"; > > allow.mount.nullfs=3D "1"; > > allow.mount.fusefs=3D "1"; ... and those extended with these allow.mount.tmpfs=3D "1"; allow.mount.linprocfs=3D "1"; > > > > Here I find the first confusing observation. I can't interact with > > the dataset > > and its content within the jail. I've set the "jailed" property of > > pool00/poudriere via "zfs set jailed=3Don pool00/poudriere" and I also= have to > > attach the jailed dataset manually via "zfs jail poudriere > > pool00/poudriere" to > > the (running) jail. But within the jail, listing ZFS's mountpoints rev= eal: > > > > NAME USED AVAIL REFER MOUNTPOINT > > pool00 124G 8.62T 34.9K /pool00 > > pool00/poudriere 34.9K 8.62T 34.9K /pool/poudriere > > > > but nothing below /pool/poudriere is visible to the jail. Being confus= ed I Since we use ezjail-admin for jail a rudimentary jail administration (just creating and/or deleting the jail, maintenance is done manually), jails ar= e rooting at pool00 /pool00 pool00/ezjail/ /pool/jails pool00/ezjail/pulverfass /pool/jails/pulverfass "pulverfass" is the jail supposed to do the poudriere's job. Since I got confused about the orientation of the "directory tree" - the r= oot is toplevel instead of downlevel - I corrected the ZFS dataset holding the poudriere stuff that way: pool00/ezjail/poudriere /pool/poudriere The jail "pulverfass" now is supposed to mount the dataset at /pool/jails/pulverfass/pool/poudriere > > Please be more verbose what you mean by "interact" and "is visible". > > Do zfs commands on the dataset work? After I corrected my mistake by not respecting the mountpoint according to statfs, with the changes explained above I'm able to mount /pool/poudriere within the jail "pulverfass", but I still have problems with the way how I= have to mount this dataset. When zfs-mounted (zfs mount -a), I'm able to use th= e dataset with poudriere as expected! But after rebooting the host and after= all jails has been restarted as well, I have to first make the dataset /pool/poudriere available to the jail via the command "zfs jail pulverfass ool00/ezjail/pulverfass" - which seems not to be done automatically by the startup process - and then from within the jail "pulverfass" I can mount t= he dataset as desribed above. This seems to be a big step ahead for me. > > Note, I don't remember if you can manage the root of the jail, but at > least subsequent jails should be possible to manage. I don't have a > jail where the root is managed in the jail, just additional ones. > Those need to have set a mountpoint after the initial jailing and then > maybe even be mounted for the first time. > > Please also check /etc/defaults/devfs.rules if the jail rule contains > an unhide entry for zfs. Within /etc/jail.conf devfs_ruleset=3D "4"; is configured as a common rulesset for all jails (in the common portion of /etc/jail.conf). There is no custom devfs.rules in /etc/, so /etc/defaults/devfs.rules shou= ld apply and as far as I can see, there is an "unhide" applied to zfs: [... /etc/defaults/devfs.rulse ...] # Devices usually found in a jail. # [devfsrules_jail=3D4] add include $devfsrules_hide_all add include $devfsrules_unhide_basic add include $devfsrules_unhide_login add path fuse unhide add path zfs unhide [...] So, I guess everything is all right from this perspective, isn't it? Is there a way to automatically provide the ZFS dataset of choice to the propper jail or do i have to either issue manually "zfs jail jailid/jailname pool/dataset" or put such a comma= nd as script-command in the jail's definition portion as exec.prestart+=3D "zfs jail ${name} pool00/ezjail/poudriere"; ? > > Bye, > Alexander. > Thank you very much and kind regards, oh