Date: Tue, 24 May 2022 13:04:06 -0600 From: Warner Losh <imp@bsdimp.com> To: mike tancsa <mike@sentex.net> Cc: freebsd-current <freebsd-current@freebsd.org> Subject: Re: make install[kernel|world] from read only /usr/obj possible ? Message-ID: <CANCZdfpF_keGwXAzFsv%2B5GwpBcmYfHDUbai--juaydLQp3RGXg@mail.gmail.com> In-Reply-To: <1cd6bcff-92d5-f155-9b19-b3a16d75fe19@sentex.net> References: <1cd6bcff-92d5-f155-9b19-b3a16d75fe19@sentex.net>
next in thread | previous in thread | raw e-mail | index | archive | help
--000000000000a0d1ba05dfc69f84
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Sorry for the top post...
It should work. It has worked in the past. This is a regression.
Warner
On Tue, May 24, 2022, 12:59 PM mike tancsa <mike@sentex.net> wrote:
> On RELENG_12, I was able to mount via NFS /usr/obj and src and do an
> installworld on read only mounts. However, with RELENG_13 and above, I
> get permission denied errors. The install seems to continue just fine,
> but I am not sure if something subtle is being missed. Looking at what
> gets installed, the kernel modules get installed just fine. However, it
> seems installworld has some problems with static libs (.a files). They
> dont seem to get re-installed. However, not sure if thats by design as
> doing a quick checksum, there is no difference on the build server and
> target of the ~ 700 .a files I checked in /usr/lib32 and /usr/lib. So
> despite the time stamps being off, maybe it just does not bother to
> update the target if they have not changed?
>
> e.g.
>
> make KERNCONF=3Dvtnet installkernel
>
>
> make warning: /usr/obj/usr/src: Permission denied.
> make[1] warning: /usr/src/: Permission denied.
> make[2] warning: /usr/obj/usr/src/amd64.amd64: Permission denied.
> --------------------------------------------------------------
> >>> Install check kernel
> --------------------------------------------------------------
> --------------------------------------------------------------
> >>> Installing kernel vtnet on Tue May 24 13:50:52 EDT 2022
> --------------------------------------------------------------
> cd /usr/obj/usr/src/amd64.amd64/sys/vtnet; MACHINE_ARCH=3Damd64
> MACHINE=3Damd64 CPUTYPE=3D CC=3D"cc -target x86_64-unknown-freebsd13.1
> --sysroot=3D/usr/obj/usr/src/amd64.amd64/tmp
> -B/usr/obj/usr/src/amd64.amd64/tmp/usr/bin" CXX=3D"c++ -target
> x86_64-unknown-freebsd13.1 --sysroot=3D/usr/obj/usr/src/amd64.amd64/tmp
> -B/usr/obj/usr/src/amd64.amd64/tmp/usr/bin" CPP=3D"cpp -target
> x86_64-unknown-freebsd13.1 --sysroot=3D/usr/obj/usr/src/amd64.amd64/tmp
> -B/usr/obj/usr/src/amd64.amd64/tmp/usr/bin" AS=3D"as" AR=3D"ar"
> ELFCTL=3D"elfctl" LD=3D"ld" LLVM_LINK=3D"" NM=3Dnm OBJCOPY=3D"objcopy"
> RANLIB=3Dranlib STRINGS=3D SIZE=3D"size" STRIPBIN=3D"strip"
> PATH=3D/usr/obj/usr/src/amd64.amd64/tmp/bin:/usr/obj/usr/src/amd64.amd64/=
tmp/usr/sbin:/usr/obj/usr/src/amd64.amd64/tmp/usr/bin:/usr/obj/usr/src/amd6=
4.amd64/tmp/legacy/usr/sbin:/usr/obj/usr/src/amd64.amd64/tmp/legacy/usr/bin=
:/usr/obj/usr/src/amd64.amd64/tmp/legacy/bin:/usr/obj/usr/src/amd64.amd64/t=
mp/legacy/usr/libexec::/sbin:/bin:/usr/sbin:/usr/bin
>
> make KERNEL=3Dkernel install
> make[2] warning: /usr/obj/usr/src/amd64.amd64/sys/vtnet: Permission denie=
d.
> thiskernel=3D`sysctl -n kern.bootfile || echo /boot/kernel/kernel` ; if =
[
> ! "`dirname "$thiskernel"`" -ef /boot/kernel ] ; then chflags -R noschg
> /boot/kernel ; rm -rf /boot/kernel ; rm -rf /usr/lib/debug/boot/kernel
> ; else if [ -d /boot/kernel.old ] ; then chflags -R noschg
> /boot/kernel.old ; rm -rf /boot/kernel.old ; fi ; mv /boot/kernel
> /boot/kernel.old ; if [ -n "/usr/lib/debug" -a -d
> /usr/lib/debug/boot/kernel ]; then rm -rf
> /usr/lib/debug/boot/kernel.old ; mv /usr/lib/debug/boot/kernel
> /usr/lib/debug/boot/kernel.old ; fi ; sysctl
> kern.bootfile=3D/boot/kernel.old/"`basename "$thiskernel"`" ; fi
> kern.bootfile: /boot/kernel/kernel -> /boot/kernel.old/kernel
> mkdir -p /boot/kernel
> install -p -m 555 -o root -g wheel kernel /boot/kernel/
> mkdir -p /usr/lib/debug/boot/kernel
> install -p -m 555 -o root -g wheel kernel.debug /usr/lib/debug/boot/kerne=
l/
> cd /usr/src/sys/modules;
> MAKEOBJDIRPREFIX=3D/usr/obj/usr/src/amd64.amd64/sys/vtnet/modules
> KMODDIR=3D/boot/kernel MACHINE_CPUARCH=3Damd64 MACHINE=3Damd64
> MACHINE_ARCH=3Damd64 MODULES_EXTRA=3D"" WITHOUT_MODULES=3D"" ARCH_FLAGS=
=3D""
> DEBUG_FLAGS=3D"-g" __MPATH=3D""
> KERNBUILDDIR=3D"/usr/obj/usr/src/amd64.amd64/sys/vtnet"
> SYSDIR=3D"/usr/src/sys" MODULE_TIED=3Dyes WITH_CTF=3D"1" KCSAN_ENABLED=3D=
"yes"
> COMPAT_FREEBSD32_ENABLED=3D"yes" make install
> make[3] warning:
> /usr/obj/usr/src/amd64.amd64/sys/vtnet/modules/usr/src/sys/modules:
> Permission denied.
> =3D=3D=3D> aac (install)
> make[4] warning:
> /usr/obj/usr/src/amd64.amd64/sys/vtnet/modules/usr/src/sys/modules/aac:
> Permission denied.
> install -T release -o root -g wheel -m 555 aac.ko /boot/kernel/
> install -T dbg -o root -g wheel -m 555 aac.ko.debug
> /usr/lib/debug/boot/kernel/
> =3D=3D=3D> aacraid (install)
> make[4] warning:
> /usr/obj/usr/src/amd64.amd64/sys/vtnet/modules/usr/src/sys/modules/aacrai=
d:
>
> Permission denied.
> install -T release -o root -g wheel -m 555 aacraid.ko /boot/kernel/
> install -T dbg -o root -g wheel -m 555 aacraid.ko.debug
> /usr/lib/debug/boot/kernel/
> =3D=3D=3D> accf_data (install)
> make[4] warning:
> /usr/obj/usr/src/amd64.amd64/sys/vtnet/modules/usr/src/sys/modules/accf_d=
ata:
>
> Permission denied.
>
>
> eg.
>
> 0{git}% ls -l /usr/lib32/libhei*
> -r--r--r-- 1 root wheel - 53622 Mar 17 20:02 /usr/lib32/libheimbase.a
> lrwxr-xr-x 1 root wheel - 17 May 24 17:11
> /usr/lib32/libheimbase.so -> libheimbase.so.11
> -r--r--r-- 1 root wheel - 13224 May 24 17:11
> /usr/lib32/libheimbase.so.11
> -r--r--r-- 1 root wheel - 54966 Mar 17 20:02 /usr/lib32/libheimbase_p.=
a
> -r--r--r-- 1 root wheel - 67598 Mar 17 20:02 /usr/lib32/libheimntlm.a
> lrwxr-xr-x 1 root wheel - 17 May 24 17:11
> /usr/lib32/libheimntlm.so -> libheimntlm.so.11
> -r--r--r-- 1 root wheel - 23840 May 24 17:11
> /usr/lib32/libheimntlm.so.11
> -r--r--r-- 1 root wheel - 68746 Mar 17 20:02 /usr/lib32/libheimntlm_p.=
a
> 0{git}%
>
>
> Looking at the pcap, nothing stands out as to what the problem might be.
> Looking at the replies, the ERROR is just "ERROR: No such file or
> directory" and doesnt correlate to the permission denied errors
>
> reading from file nfs.pcap, link-type EN10MB (Ethernet)
> 2 xid reply ok
> 3371 xid reply ok getattr CHR 11734062556 ids
> 951 xid reply ok getattr CHR 12231260544 ids
> 626 xid reply ok getattr ERROR: No such
> 2 xid reply ok getattr FIFO 10432271160 ids
> 6758 xid reply ok getattr FIFO 10731272141 ids
> 165 xid reply ok getattr FIFO 11433667553 ids
> 214 xid reply ok getattr FIFO 12231260544 ids
> 3373 xid reply ok getattr LNK 10333067563 ids
> 2 xid reply ok getattr LNK 12231267145 ids
> 2 xid reply ok getattr unk-ft 10 10536061550
> 2 xid reply ok getattr unk-ft 13 10334462541
> 2 xid reply ok getattr unk-ft 15 12231261554
> 673 xid reply ok getattr unk-ft 6 10130661545
> 879 xid reply ok getattr unk-ft 6 11433667553
> 126 xid reply ok getattr unk-ft 8 12231260544
> 2 xid reply ok null
>
>
>
>
--000000000000a0d1ba05dfc69f84
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
<div dir=3D"auto">Sorry for the top post...<div dir=3D"auto"><br></div><div=
dir=3D"auto">It should work. It has worked in the past. This is a regressi=
on.=C2=A0</div><div dir=3D"auto"><br></div><div dir=3D"auto">Warner=C2=A0</=
div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_at=
tr">On Tue, May 24, 2022, 12:59 PM mike tancsa <<a href=3D"mailto:mike@s=
entex.net">mike@sentex.net</a>> wrote:<br></div><blockquote class=3D"gma=
il_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-lef=
t:1ex">On RELENG_12, I was able to mount via NFS /usr/obj and src and do an=
<br>
installworld on read only mounts. However, with RELENG_13 and above, I <br>
get permission denied errors. The install seems to continue just fine, <br>
but I am not sure if something subtle is being missed. Looking at what <br>
gets installed, the kernel modules get installed just fine.=C2=A0 However, =
it <br>
seems installworld has some problems with static libs (.a files). They <br>
dont seem to get re-installed. However, not sure if thats by design as <br>
doing a quick checksum, there is no difference on the build server and <br>
target of the ~ 700 .a files I checked in /usr/lib32 and /usr/lib. So <br>
despite the time stamps being off, maybe it just does not bother to <br>
update the target if they have not changed?<br>
<br>
e.g.<br>
<br>
make KERNCONF=3Dvtnet installkernel<br>
<br>
<br>
make warning: /usr/obj/usr/src: Permission denied.<br>
make[1] warning: /usr/src/: Permission denied.<br>
make[2] warning: /usr/obj/usr/src/amd64.amd64: Permission denied.<br>
--------------------------------------------------------------<br>
=C2=A0>>> Install check kernel<br>
--------------------------------------------------------------<br>
--------------------------------------------------------------<br>
=C2=A0>>> Installing kernel vtnet on Tue May 24 13:50:52 EDT 2022<=
br>
--------------------------------------------------------------<br>
cd /usr/obj/usr/src/amd64.amd64/sys/vtnet;=C2=A0 MACHINE_ARCH=3Damd64 <br>
MACHINE=3Damd64=C2=A0 CPUTYPE=3D CC=3D"cc -target x86_64-unknown-freeb=
sd13.1 <br>
--sysroot=3D/usr/obj/usr/src/amd64.amd64/tmp <br>
-B/usr/obj/usr/src/amd64.amd64/tmp/usr/bin" CXX=3D"c++=C2=A0 -tar=
get <br>
x86_64-unknown-freebsd13.1 --sysroot=3D/usr/obj/usr/src/amd64.amd64/tmp <br=
>
-B/usr/obj/usr/src/amd64.amd64/tmp/usr/bin"=C2=A0 CPP=3D"cpp -tar=
get <br>
x86_64-unknown-freebsd13.1 --sysroot=3D/usr/obj/usr/src/amd64.amd64/tmp <br=
>
-B/usr/obj/usr/src/amd64.amd64/tmp/usr/bin"=C2=A0 AS=3D"as" =
AR=3D"ar" <br>
ELFCTL=3D"elfctl" LD=3D"ld"=C2=A0 LLVM_LINK=3D"&qu=
ot; NM=3Dnm OBJCOPY=3D"objcopy" <br>
RANLIB=3Dranlib STRINGS=3D=C2=A0 SIZE=3D"size" STRIPBIN=3D"s=
trip" <br>
PATH=3D/usr/obj/usr/src/amd64.amd64/tmp/bin:/usr/obj/usr/src/amd64.amd64/tm=
p/usr/sbin:/usr/obj/usr/src/amd64.amd64/tmp/usr/bin:/usr/obj/usr/src/amd64.=
amd64/tmp/legacy/usr/sbin:/usr/obj/usr/src/amd64.amd64/tmp/legacy/usr/bin:/=
usr/obj/usr/src/amd64.amd64/tmp/legacy/bin:/usr/obj/usr/src/amd64.amd64/tmp=
/legacy/usr/libexec::/sbin:/bin:/usr/sbin:/usr/bin <br>
make=C2=A0 KERNEL=3Dkernel install<br>
make[2] warning: /usr/obj/usr/src/amd64.amd64/sys/vtnet: Permission denied.=
<br>
thiskernel=3D`sysctl -n kern.bootfile || echo /boot/kernel/kernel` ;=C2=A0 =
if [ <br>
! "`dirname "$thiskernel"`" -ef /boot/kernel ] ; then c=
hflags -R noschg <br>
/boot/kernel ;=C2=A0 rm -rf /boot/kernel ;=C2=A0 rm -rf /usr/lib/debug/boot=
/kernel <br>
;=C2=A0 else=C2=A0 if [ -d /boot/kernel.old ] ; then=C2=A0 chflags -R nosch=
g <br>
/boot/kernel.old ;=C2=A0 rm -rf /boot/kernel.old ;=C2=A0 fi ;=C2=A0 mv /boo=
t/kernel <br>
/boot/kernel.old ;=C2=A0 if [ -n "/usr/lib/debug" -a=C2=A0 -d <br=
>
/usr/lib/debug/boot/kernel ]; then=C2=A0 rm -rf <br>
/usr/lib/debug/boot/kernel.old ;=C2=A0 mv /usr/lib/debug/boot/kernel <br>
/usr/lib/debug/boot/kernel.old ;=C2=A0 fi ; sysctl <br>
kern.bootfile=3D/boot/kernel.old/"`basename "$thiskernel"`&q=
uot; ;=C2=A0 fi<br>
kern.bootfile: /boot/kernel/kernel -> /boot/kernel.old/kernel<br>
mkdir -p /boot/kernel<br>
install -p -m 555 -o root -g wheel kernel /boot/kernel/<br>
mkdir -p /usr/lib/debug/boot/kernel<br>
install -p -m 555 -o root -g wheel kernel.debug /usr/lib/debug/boot/kernel/=
<br>
cd /usr/src/sys/modules; <br>
MAKEOBJDIRPREFIX=3D/usr/obj/usr/src/amd64.amd64/sys/vtnet/modules <br>
KMODDIR=3D/boot/kernel MACHINE_CPUARCH=3Damd64 MACHINE=3Damd64 <br>
MACHINE_ARCH=3Damd64 MODULES_EXTRA=3D"" WITHOUT_MODULES=3D"&=
quot; ARCH_FLAGS=3D"" <br>
DEBUG_FLAGS=3D"-g" __MPATH=3D"" <br>
KERNBUILDDIR=3D"/usr/obj/usr/src/amd64.amd64/sys/vtnet" <br>
SYSDIR=3D"/usr/src/sys" MODULE_TIED=3Dyes WITH_CTF=3D"1"=
; KCSAN_ENABLED=3D"yes" <br>
COMPAT_FREEBSD32_ENABLED=3D"yes" make=C2=A0 install<br>
make[3] warning: <br>
/usr/obj/usr/src/amd64.amd64/sys/vtnet/modules/usr/src/sys/modules: <br>
Permission denied.<br>
=3D=3D=3D> aac (install)<br>
make[4] warning: <br>
/usr/obj/usr/src/amd64.amd64/sys/vtnet/modules/usr/src/sys/modules/aac: <br=
>
Permission denied.<br>
install -T release -o root -g wheel -m 555=C2=A0=C2=A0 aac.ko /boot/kernel/=
<br>
install -T dbg -o root -g wheel -m 555=C2=A0=C2=A0 aac.ko.debug <br>
/usr/lib/debug/boot/kernel/<br>
=3D=3D=3D> aacraid (install)<br>
make[4] warning: <br>
/usr/obj/usr/src/amd64.amd64/sys/vtnet/modules/usr/src/sys/modules/aacraid:=
<br>
Permission denied.<br>
install -T release -o root -g wheel -m 555=C2=A0=C2=A0 aacraid.ko /boot/ker=
nel/<br>
install -T dbg -o root -g wheel -m 555=C2=A0=C2=A0 aacraid.ko.debug <br>
/usr/lib/debug/boot/kernel/<br>
=3D=3D=3D> accf_data (install)<br>
make[4] warning: <br>
/usr/obj/usr/src/amd64.amd64/sys/vtnet/modules/usr/src/sys/modules/accf_dat=
a: <br>
Permission denied.<br>
<br>
<br>
eg.<br>
<br>
0{git}% ls -l /usr/lib32/libhei*<br>
-r--r--r--=C2=A0 1 root=C2=A0 wheel=C2=A0 - 53622 Mar 17 20:02 /usr/lib32/l=
ibheimbase.a<br>
lrwxr-xr-x=C2=A0 1 root=C2=A0 wheel=C2=A0 -=C2=A0=C2=A0=C2=A0 17 May 24 17:=
11 <br>
/usr/lib32/libheimbase.so -> libheimbase.so.11<br>
-r--r--r--=C2=A0 1 root=C2=A0 wheel=C2=A0 - 13224 May 24 17:11 /usr/lib32/l=
ibheimbase.so.11<br>
-r--r--r--=C2=A0 1 root=C2=A0 wheel=C2=A0 - 54966 Mar 17 20:02 /usr/lib32/l=
ibheimbase_p.a<br>
-r--r--r--=C2=A0 1 root=C2=A0 wheel=C2=A0 - 67598 Mar 17 20:02 /usr/lib32/l=
ibheimntlm.a<br>
lrwxr-xr-x=C2=A0 1 root=C2=A0 wheel=C2=A0 -=C2=A0=C2=A0=C2=A0 17 May 24 17:=
11 <br>
/usr/lib32/libheimntlm.so -> libheimntlm.so.11<br>
-r--r--r--=C2=A0 1 root=C2=A0 wheel=C2=A0 - 23840 May 24 17:11 /usr/lib32/l=
ibheimntlm.so.11<br>
-r--r--r--=C2=A0 1 root=C2=A0 wheel=C2=A0 - 68746 Mar 17 20:02 /usr/lib32/l=
ibheimntlm_p.a<br>
0{git}%<br>
<br>
<br>
Looking at the pcap, nothing stands out as to what the problem might be. <b=
r>
Looking at the replies, the ERROR is just "ERROR: No such file or <br>
directory" and doesnt correlate to the permission denied errors<br>
<br>
reading from file nfs.pcap, link-type EN10MB (Ethernet)<br>
=C2=A0=C2=A0=C2=A0 2 xid=C2=A0 reply ok<br>
3371 xid=C2=A0 reply ok getattr CHR 11734062556 ids<br>
=C2=A0=C2=A0951 xid=C2=A0 reply ok getattr CHR 12231260544 ids<br>
=C2=A0=C2=A0626 xid=C2=A0 reply ok getattr ERROR: No such<br>
=C2=A0=C2=A0=C2=A0 2 xid=C2=A0 reply ok getattr FIFO 10432271160 ids<br>
6758 xid=C2=A0 reply ok getattr FIFO 10731272141 ids<br>
=C2=A0=C2=A0165 xid=C2=A0 reply ok getattr FIFO 11433667553 ids<br>
=C2=A0=C2=A0214 xid=C2=A0 reply ok getattr FIFO 12231260544 ids<br>
3373 xid=C2=A0 reply ok getattr LNK 10333067563 ids<br>
=C2=A0=C2=A0=C2=A0 2 xid=C2=A0 reply ok getattr LNK 12231267145 ids<br>
=C2=A0=C2=A0=C2=A0 2 xid=C2=A0 reply ok getattr unk-ft 10 10536061550<br>
=C2=A0=C2=A0=C2=A0 2 xid=C2=A0 reply ok getattr unk-ft 13 10334462541<br>
=C2=A0=C2=A0=C2=A0 2 xid=C2=A0 reply ok getattr unk-ft 15 12231261554<br>
=C2=A0=C2=A0673 xid=C2=A0 reply ok getattr unk-ft 6 10130661545<br>
=C2=A0=C2=A0879 xid=C2=A0 reply ok getattr unk-ft 6 11433667553<br>
=C2=A0=C2=A0126 xid=C2=A0 reply ok getattr unk-ft 8 12231260544<br>
=C2=A0=C2=A0=C2=A0 2 xid=C2=A0 reply ok null<br>
<br>
<br>
<br>
</blockquote></div>
--000000000000a0d1ba05dfc69f84--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CANCZdfpF_keGwXAzFsv%2B5GwpBcmYfHDUbai--juaydLQp3RGXg>