From owner-freebsd-stable@freebsd.org Tue Apr 30 11:24:16 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6B1F9158D5CD; Tue, 30 Apr 2019 11:24:16 +0000 (UTC) (envelope-from mj-mailinglist@gmx.de) Received: from mout.gmx.net (mout.gmx.net [212.227.17.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DEFB16B6E4; Tue, 30 Apr 2019 11:24:14 +0000 (UTC) (envelope-from mj-mailinglist@gmx.de) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1556623444; bh=C48Y2DAFDvdSncQPnRp9EYT/pMwLrRMv8yayxyDZlIU=; h=X-UI-Sender-Class:From:To:Subject:Date; b=Vbmz8EC0XPWBxF8fDBSybwT1pcjBarTJcrcew6W1uANckhbJ8PcJQjvdfIK/tzvFR PRp1zfBSng4KGBGUk+9YqxQ2V7FTX1htiNUCjoSFDo0MwH+T0C4465gSysCMLd8VFP oGZVPDE6wSylwNf+BRldTgKSBjOvmkDGpUWN49CU= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from [194.59.36.88] ([194.59.36.88]) by web-mail.gmx.net (3c-app-gmx-bs07.server.lan [172.19.170.56]) (via HTTP); Tue, 30 Apr 2019 13:24:04 +0200 MIME-Version: 1.0 Message-ID: From: "Martin Jakob" To: freebsd-pkgbase@freebsd.org, freebsd-stable@freebsd.org Subject: Creating jails from pkgbase packages Content-Type: text/plain; charset=UTF-8 Date: Tue, 30 Apr 2019 13:24:04 +0200 Importance: normal Sensitivity: Normal Content-Transfer-Encoding: quoted-printable X-Priority: 3 X-Provags-ID: V03:K1:tLU9i8MVgGDKBk2v80XuLSUrQJhK5X/hN4cPMCASgMLS+yH6CSD4AHSf5VDYu8vw/a9A0 WkV/5UwVeeCCKbvb1+Rlm1HZ7V0mkqret1zFY9OFthT5j6iqusnI82jTjkknr9jMRxn9IK9PvWyc +4L8HJSPSfZXtt1qz7sUuJF+Oy57kQXPSZYebSRw4HGhO85S+kdCIVI/OHNLy2rzx3ISZzeK13ue c1VNgwdpBT+8kYqt2tntRWYKQSa/1DCWucY8aLpbuH/lB9QWwzzd7I8teiwyjj5MpaGMHxOviJaD SY= X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:j6NES14qW/Q=:F5EETivzkJXM15DfCvMA7z fgKbYi/Prx72FrWmT6wQxalUYmUKQQg5u+eGpeVR8FWQFvzwQs9nMykas8NpUPCRUWXcV+ox6 0ZwCRektshKyl+HLbU7pPIjU7v4io7NZ9v4/hYWbArRklexs79K7WopcUg0wAD8qmgAdONWmA h1dATRYzzuH5OJU8ouVtzff+3E0lgAinErhRNT4UxjbVtxdm+czh0VjB4choemWxSncXbLnNR MU09f3QpVR5VGHKbGUxZ1cKGzzRIsXbkLe7MwK5HL8tfgD4D9LERdD2raK0F/kkfx/EnCdYIE C4iFnRr3W/2IKYxIK7BAZuxuZzAkT320MjS0qvQXsUPjsaT8/MbEtF1Vu75cniQNEcoVYT8tt 6qwe0P2pIUJ86BKx6Tpx4ot3GedLfjxatz10VIqVFJoLL1txjBhABiqgGMV4OdrgwyTZEYS1+ fQo28klv9QY/Dnq7n9tlm5F2RxYmbGuUoEhWj7zMyu0eXJ1X0RFHaf9Qsh8/6UZDkdNWQMxvj ASQLmSLpHOeXIiBwVbhx2bhAOUL1xVXAEoR4u2Bm5N2L+noi4mRjLDhixuSU4vS3zNQado3LH 6D+cbevu9F8ZRIodqfqEZMXKdmKN4B376+cGPiKhCMY/K76sVqMJziKw== X-Rspamd-Queue-Id: DEFB16B6E4 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmx.net header.s=badeba3b8450 header.b=Vbmz8EC0; spf=pass (mx1.freebsd.org: domain of mj-mailinglist@gmx.de designates 212.227.17.22 as permitted sender) smtp.mailfrom=mj-mailinglist@gmx.de X-Spamd-Result: default: False [-3.86 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; R_DKIM_ALLOW(-0.20)[gmx.net:s=badeba3b8450]; RCVD_IN_DNSWL_LOW(-0.10)[22.17.227.212.list.dnswl.org : 127.0.3.1]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:212.227.17.0/27]; FREEMAIL_FROM(0.00)[gmx.de]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; DMARC_NA(0.00)[gmx.de]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_TLS_LAST(0.00)[]; MX_GOOD(-0.01)[cached: mx00.emig.gmx.net]; DKIM_TRACE(0.00)[gmx.net:+]; HAS_X_PRIO_THREE(0.00)[3]; NEURAL_HAM_SHORT(-0.90)[-0.897,0]; RCPT_COUNT_TWO(0.00)[2]; IP_SCORE(-0.85)[ip: (-4.73), ipnet: 212.227.0.0/16(-1.40), asn: 8560(1.87), country: DE(-0.01)]; FROM_EQ_ENVFROM(0.00)[]; MID_RHS_NOT_FQDN(0.50)[]; FREEMAIL_ENVFROM(0.00)[gmx.de]; ASN(0.00)[asn:8560, ipnet:212.227.0.0/16, country:DE]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2019 11:24:16 -0000 My first attempt seems to be empty when its arrived at the maillinglist, so= i try it again in pure text format (i hope)=2E Sorry, if there are unwante= d duplicates=2E=20 Since a few days i was experimenting with jails build from pkgbase (i got = the idea from Michael W=2E Lucas Freebsd Jails book)=2E Now there is some d= iscussion about pkgbase again :) So i want to share my experience, i hope this is not offtopic to this conv= ersation and my writing style is not too offputting=2E=2E=2E =C2=A0 This experiments were made on an resently updated 12-stable system=2E Sorr= y, i dont have a system with head=C2=A0available right now=2E =C2=A0 Anyway, this is what i did: - I first set the variable "REPODIR=3D/usr/repo" in /etc/make=2Econf, to h= ave a nonvolatile repository (is this the right place/variable to do this?)= =2E - then after the standard "make buildworld|installworld|buildkernel|instal= lkernel" in /usr/src, i created the pkgbase packages with the command "make= packages"=2E =C2=A0 the files are placed in "/usr/repo/FreeBSD:12:amd64/latest" where l= atest is a symlink to the, well latest build=2E =C2=A0 drwxr-xr-x =C2=A02 root =C2=A0wheel =C2=A0805 14 Apr=2E 14:59 12=2E= 0=2Es20190414123806/ =C2=A0 drwxr-xr-x =C2=A02 root =C2=A0wheel =C2=A0805 21 Apr=2E 09:27 12=2E= 0=2Es20190419155807/ =C2=A0 drwxr-xr-x =C2=A02 root =C2=A0wheel =C2=A0805 22 Apr=2E 13:53 12=2E= 0=2Es20190422094219/ =C2=A0 drwxr-xr-x =C2=A02 root =C2=A0wheel =C2=A0805 27 Apr=2E 14:12 12=2E= 0=2Es20190427084356/ =C2=A0 lrwxr-xr-x =C2=A01 root =C2=A0wheel =C2=A0 20 27 Apr=2E 11:04 lates= t@ -> 12=2E0=2Es20190427084356 =C2=A0=C2=A0 - this repo is published via the file "/usr/local/etc/pkg/repos/FreeBSD-ba= se=2Econf"which has this content: root@betablock:~ # cat "/usr/local/etc/pkg/repos/FreeBSD-base=2Econf" # FreeBSD base system repository FreeBSD-base: { =C2=A0 url: "file:///usr/repo/${ABI}/latest", =C2=A0 mirror_type: "none", =C2=A0 enabled: yes } =C2=A0 now to the jail stuff: - this command installs the FreeBSD-runtime and the 72 neccesary dependenc= ies in the jails root directory /jails/test03: root@betablock:~ # pkg --rootdir /jails/test03 -o 'ASSUME_ALWAYS_YES=3Dtru= e' install -r FreeBSD-base FreeBSD-runtime =C2=A0 - the newly created jail "test03" (an entry in /etc/jail=2Econf already ex= ists) is started with the command: root@betablock:~ # jail -vc test03 =C2=A0 sadly the jail wont start, it exits with this message: =2E=2E=2E test03: created test03: run command in jail: /bin/sh /etc/rc jail: test03: getpwnam: No such file or directory jail: test03: /bin/sh /etc/rc: failed test03: removed =2E=2E=2E =C2=A0 This happens, because there are no user-database files (like master=2Epass= wd, pwd=2Edb, spwd=2Edb) and no group files in the jails /etc/ directory=2E= hmmm=2E=2E=2E I saw in the pkg install output of the FreeBSD-runtime =C2=A0pkg this mess= age: =2E=2E=2E [2/72] Extracting FreeBSD-runtime-12=2E0=2Es20190414123806: 100% pwd_mkdb: /jails/test03/etc/master=2Epasswd: No such file or directory =2E=2E=2E so pkg tries to run the pwd_mkdb as defined in the runtime=2Eucl file, or = embedded in the pkg file, it is viewable with this command: root@betablock:~ # pkg info --raw --file /usr/repo/FreeBSD:12:amd64/latest= /FreeBSD-runtime-12=2E0=2Es20190414123806=2Etxz =2E=2E=2E scripts: { =C2=A0 =C2=A0 post-install: "cap_mkdb -l ${PKG_ROOTDIR}/etc/login=2Econf\n= \tpwd_mkdb -i -p -d =C2=A0${PKG_ROOTDIR}/etc ${PKG_ROOTDIR}/etc/master=2Epa= sswd\n\tservices_mkdb -l -q -o ${PKG_ROOTDIR}/var/db/services=2Edb ${PKG_RO= OTDIR}/etc/services\n\tchmod 1777 ${PKG_ROOTDIR}/tmp" } =2E=2E=2E - so for a quick fix, i copied the "missing" files (master=2Epasswd, pwd= =2Edb, spwd=2Edb, group) from a FreeBSD base=2Etxz package to my jails etc = directory and tried to start the jail again=2E Read below for a "better" so= lution=2E Now the jail starts: root@betablock:~ # jls =C2=A0 =C2=A0 =C2=A0JID =C2=A0IP Address =C2=A0 =C2=A0 =C2=A0Hostname =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Pa= th =C2=A0 =C2=A0 =C2=A0=2E=2E=2E =C2=A0 =C2=A0 =C2=A05 =C2=A0192=2E168=2E0=2E113 =C2=A0 test03=2Elocal =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0/jails/test03 =C2=A0 i can "enter" it with this command: root@betablock:~ # jexec -l test03 =C2=A0=C2=A0 now to some more experiments with the jail which results in some questions= (also, see below) =C2=A0 - trying to ping something: root@test03:~ # ping google=2Ecom ld-elf=2Eso=2E1: Shared object "libcap_dns=2Eso=2E1" not found, required b= y "ping" =C2=A0 the required library libcap_dns=2Eso=2E1 is in the file FreeBSD-libcasper-= 12=2E0=2Es20190414123806=2Etxz, so lets install it: Outside of the jail: root@betablock:~ # pkg --rootdir /jails/test03 -o 'ASSUME_ALWAYS_YES=3Dtru= e' install -r FreeBSD-base FreeBSD-libcasper =C2=A0 and inside the jail: root@test03:~ # ping google=2Ecom PING google=2Ecom (216=2E58=2E213=2E206): 56 data bytes 64 bytes from 216=2E58=2E213=2E206: icmp_seq=3D0 ttl=3D52 time=3D23=2E398 = ms 64 bytes from 216=2E58=2E213=2E206: icmp_seq=3D1 ttl=3D52 time=3D23=2E975 = ms 64 bytes from 216=2E58=2E213=2E206: icmp_seq=3D2 ttl=3D52 time=3D23=2E252 = ms =2E=2E=2E success! =C2=A0 The size of this freshly installed jail: du -h -d 1 /jails/ =2E=2E=2E 90M =C2=A0 =C2=A0/jails/test03 =C2=A0 - now lets install some packages from outside of the jail: root@betablock:~ # pkg --rootdir /jails/test03 install nginx-full Updating FreeBSD repository catalogue=2E=2E=2E pkg: Repository FreeBSD load error: access repo file(/jails/test03/var/db/= pkg/repo-FreeBSD=2Esqlite) failed: No such file or directory Fetching meta=2Etxz: 100% =C2=A0 =C2=A0944 B =C2=A0 0=2E9kB/s =C2=A0 =C2= =A000:01 Fetching packagesite=2Etxz: 100% =C2=A0 =C2=A06 MiB =C2=A0 2=2E2MB/s =C2= =A0 =C2=A000:03 Processing entries: 100% FreeBSD repository update completed=2E 31883 packages processed=2E Updating Synth repository catalogue=2E=2E=2E Synth repository is up to date=2E Updating FreeBSD-base repository catalogue=2E=2E=2E FreeBSD-base repository is up to date=2E All repositories are up to date=2E The following 146 package(s) will be affected (of 0 checked): New packages to be INSTALLED: =C2=A0 =C2=A0 =C2=A0 =C2=A0 nginx-full: 1=2E16=2E0_2,2 [FreeBSD] =C2=A0 =C2=A0 =C2=A0 =C2=A0 apache24: 2=2E4=2E39 [FreeBSD] =C2=A0 =C2=A0 =C2=A0 =C2=A0 libnghttp2: 1=2E38=2E0 [FreeBSD] =2E=2E=2E =C2=A0 =C2=A0 =C2=A0 =C2=A0 libdrizzle: 0=2E8_6 [FreeBSD] =C2=A0 =C2=A0 =C2=A0 =C2=A0 brotli: 1=2E0=2E7_1,1 [FreeBSD] Number of packages to be installed: 146 The process will require 953 MiB more space=2E 232 MiB to be downloaded=2E =2E=2E=2E =C2=A0 Funfact -=C2=A0the jailsize after this installation: 1,0G =C2=A0 =C2=A0/jails/test03 =C2=A0 - lets enable the nginx service root@betablock:~ # sysrc -R /jails/test03 nginx_enable=3DYES nginx_enable: =C2=A0-> YES =C2=A0 - and start it (after a jail restart, without it nginx does not find its l= ibraries, installed above) root@test03:~ # service nginx start Performing sanity check on nginx configuration: ld-elf=2Eso=2E1: Shared object "libexecinfo=2Eso=2E1" not found, required = by "libprofiler=2Eso=2E0" =C2=A0 - lets install this missing library with the command=C2=A0 root@betablock:~ # pkg --rootdir /jails/test03 -o 'ASSUME_ALWAYS_YES=3Dtru= e' install -r FreeBSD-base FreeBSD-libexecinfo =C2=A0 - and lets also install vi: root@betablock:~ # pkg --rootdir /jails/test03 -o 'ASSUME_ALWAYS_YES=3Dtru= e' install -r FreeBSD-base FreeBSD-vi =C2=A0 - so lets try again (after creating the missing config files nginx=2Econf = and mime=2Etypes): root@betablock:~ # jexec -l test03 service nginx start Starting nginx=2E nginx: [emerg] gethostbyname() failed in /usr/local/etc/nginx/nginx=2Econf= :122 =C2=A0 - and the check: root@betablock:~ # jexec -l test03 service nginx status nginx is running as pid 41359=2E =C2=A0 (Browser access works too=2E=2E=2E) =C2=A0 - Now Lets work with packages within the jail: root@test03:~ # pkg update The package management tool is not yet installed on your system=2E Do you want to fetch and install it now? [y/N]: y Bootstrapping pkg from pkg+http://pkg=2EFreeBSD=2Eorg/FreeBSD:12:amd64/lat= est, please wait=2E=2E=2E pkg: Error loading revoked certificates =C2=A0 - Hm, this is because the directory "/usr/share/keys/pkg/revoked/" is miss= ing, lets create it: root@test03:~ # mkdir /usr/share/keys/pkg/revoked/ =C2=A0 - and now: root@test03:~ # pkg update The package management tool is not yet installed on your system=2E Do you want to fetch and install it now? [y/N]: y Bootstrapping pkg from pkg+http://pkg=2EFreeBSD=2Eorg/FreeBSD:12:amd64/lat= est[http://pkg=2EFreeBSD=2Eorg/FreeBSD:12:amd64/latest], please wait=2E=2E= =2E Verifying signature with trusted certificate pkg=2Efreebsd=2Eorg=2E2013102= 301=2E=2E=2E done [test03=2Elocal] Installing pkg-1=2E10=2E5_5=2E=2E=2E [test03=2Elocal] Extracting pkg-1=2E10=2E5_5: 100% Updating FreeBSD repository catalogue=2E=2E=2E FreeBSD repository is up to date=2E All repositories are up to date=2E root@test03:~ # pkg info FreeBSD-casper-12=2E0=2Es20190414123806 casper package FreeBSD-clibs-12=2E0=2Es20190414123806 Core C Libraries FreeBSD-jail-12=2E0=2Es20190414123806 Jail Utilities FreeBSD-lib-12=2E0=2Es20190414123806 lib package FreeBSD-lib80211-12=2E0=2Es20190414123806 lib80211 package =2E=2E=2E x265-3=2E0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 H=2E265/High Efficiency Video Coding (HEVC) format xorg-fonts-truetype-7=2E7_1 =C2=A0 =C2=A0 =C2=A0X=2EOrg TrueType fonts xorgproto-2018=2E4 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 xorg p= rotocol headers xvid-1=2E3=2E5,1 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 Opensource MPEG-4 codec, based on OpenDivx yajl-2=2E1=2E0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 Portable JSON parsing and serialization library in ANSI C Btw=2E The missing "revoked" directory problem shoud be fixed with https:/= /reviews=2Efreebsd=2Eorg/D20056[https://reviews=2Efreebsd=2Eorg/D20056] --- So this is a basic jail installation with pkgbase, now to some further exp= eriments and resulting questions: =C2=A0 - i looked into why the userdatabase was missing in the newly created jail= =2E Well, it is, because the files are not included in the default Freebsd-= runtime package=2E =C2=A0 To check if it would work, i built my own Freebsd-runtime package w= ith these files added=2E to do this, i copied the runtime=2Eplist file from= the $WSTAGEDIR and modified it=2E =C2=A0 - first i removed a lot of duplicated directory entries like =C2=A0 @dir(root,wheel,0755,) /boot =C2=A0 @dir(root,wheel,0755,) /boot =C2=A0 @dir(root,wheel,0755,) /boot =C2=A0 @dir(root,wheel,0755,) /boot =C2=A0 =2E=2E=2E (and many more) =C2=A0 - then i added the "missing" files and directory =C2=A0 @config(root,wheel,0600,) /etc/master=2Epasswd =C2=A0 @config(root,wheel,0644,) /etc/group =C2=A0 @dir(root,wheel,0755,) /usr/share/keys/pkg/revoked =C2=A0=C2=A0 =C2=A0 - after that i rebuild the Freebsd-runtime package with these chang= es (found in /usr/src/Makefile=2Einc1): =C2=A0 root@betablock:~ # pkg -o ABI_FILE=3D/usr/obj/usr/src/amd64=2Eamd64= /worldstage/usr/bin/uname -o ALLOW_BASE_SHLIBS=3Dyes create -M /usr/obj/usr= /src/amd64=2Eamd64/worldstage/runtime=2Eucl -p /root/pkgs/runtime=2Eplist -= r /usr/obj/usr/src/amd64=2Eamd64/worldstage --output-dir /usr/repo/FreeBSD:= 12:amd64/latest =C2=A0=C2=A0 =C2=A0 and rebuilding the repo metadata with: =C2=A0 root@betablock:~ # pkg repo /usr/repo/FreeBSD:12:amd64/latest =C2=A0 a jail created with this Freebsd-runtime starts directly, without t= he steps mentioned above=2E =C2=A0=C2=A0 =C2=A0 My Question: Is the ommission of these files intentional?=C2=A0 =C2=A0=C2=A0 =C2=A0 - one last experiment:=C2=A0i deleted the manpages, the examples, a= nd the /boot directory (i guessed a jail has no need for it), the jail buil= d from this file is 75M, the installation of packages like nginx=C2=A0works= like in the original file=2E =C2=A0=C2=A0 =C2=A0 - should the FreeBSD-libcasper package be installed as a dependency= of the FreeBSD-casper package? =C2=A0 =C2=A0 - i did not observe more missing libraries like libcasper above, bu= t depending on the tasks that are=C2=A0executed or packages that are instal= led, there might be more that are missing=2E =C2=A0 So that was my pkgbase-jail experiment, it was fun and i learned some thin= gs about pkgbase=2E Any comments, tips, tricks and (mild) critique are welc= ome=2E =C2=A0 Greetings =C2=A0 Martin =C2=A0