Date: Tue, 24 May 2022 13:04:06 -0600 From: Warner Losh <imp@bsdimp.com> To: mike tancsa <mike@sentex.net> Cc: freebsd-current <freebsd-current@freebsd.org> Subject: Re: make install[kernel|world] from read only /usr/obj possible ? Message-ID: <CANCZdfpF_keGwXAzFsv%2B5GwpBcmYfHDUbai--juaydLQp3RGXg@mail.gmail.com> In-Reply-To: <1cd6bcff-92d5-f155-9b19-b3a16d75fe19@sentex.net>
index | next in thread | previous in thread | raw e-mail
[-- Attachment #1 --]
Sorry for the top post...
It should work. It has worked in the past. This is a regression.
Warner
On Tue, May 24, 2022, 12:59 PM mike tancsa <mike@sentex.net> wrote:
> On RELENG_12, I was able to mount via NFS /usr/obj and src and do an
> installworld on read only mounts. However, with RELENG_13 and above, I
> get permission denied errors. The install seems to continue just fine,
> but I am not sure if something subtle is being missed. Looking at what
> gets installed, the kernel modules get installed just fine. However, it
> seems installworld has some problems with static libs (.a files). They
> dont seem to get re-installed. However, not sure if thats by design as
> doing a quick checksum, there is no difference on the build server and
> target of the ~ 700 .a files I checked in /usr/lib32 and /usr/lib. So
> despite the time stamps being off, maybe it just does not bother to
> update the target if they have not changed?
>
> e.g.
>
> make KERNCONF=vtnet installkernel
>
>
> make warning: /usr/obj/usr/src: Permission denied.
> make[1] warning: /usr/src/: Permission denied.
> make[2] warning: /usr/obj/usr/src/amd64.amd64: Permission denied.
> --------------------------------------------------------------
> >>> Install check kernel
> --------------------------------------------------------------
> --------------------------------------------------------------
> >>> Installing kernel vtnet on Tue May 24 13:50:52 EDT 2022
> --------------------------------------------------------------
> cd /usr/obj/usr/src/amd64.amd64/sys/vtnet; MACHINE_ARCH=amd64
> MACHINE=amd64 CPUTYPE= CC="cc -target x86_64-unknown-freebsd13.1
> --sysroot=/usr/obj/usr/src/amd64.amd64/tmp
> -B/usr/obj/usr/src/amd64.amd64/tmp/usr/bin" CXX="c++ -target
> x86_64-unknown-freebsd13.1 --sysroot=/usr/obj/usr/src/amd64.amd64/tmp
> -B/usr/obj/usr/src/amd64.amd64/tmp/usr/bin" CPP="cpp -target
> x86_64-unknown-freebsd13.1 --sysroot=/usr/obj/usr/src/amd64.amd64/tmp
> -B/usr/obj/usr/src/amd64.amd64/tmp/usr/bin" AS="as" AR="ar"
> ELFCTL="elfctl" LD="ld" LLVM_LINK="" NM=nm OBJCOPY="objcopy"
> RANLIB=ranlib STRINGS= SIZE="size" STRIPBIN="strip"
> PATH=/usr/obj/usr/src/amd64.amd64/tmp/bin:/usr/obj/usr/src/amd64.amd64/tmp/usr/sbin:/usr/obj/usr/src/amd64.amd64/tmp/usr/bin:/usr/obj/usr/src/amd64.amd64/tmp/legacy/usr/sbin:/usr/obj/usr/src/amd64.amd64/tmp/legacy/usr/bin:/usr/obj/usr/src/amd64.amd64/tmp/legacy/bin:/usr/obj/usr/src/amd64.amd64/tmp/legacy/usr/libexec::/sbin:/bin:/usr/sbin:/usr/bin
>
> make KERNEL=kernel install
> make[2] warning: /usr/obj/usr/src/amd64.amd64/sys/vtnet: Permission denied.
> thiskernel=`sysctl -n kern.bootfile || echo /boot/kernel/kernel` ; if [
> ! "`dirname "$thiskernel"`" -ef /boot/kernel ] ; then chflags -R noschg
> /boot/kernel ; rm -rf /boot/kernel ; rm -rf /usr/lib/debug/boot/kernel
> ; else if [ -d /boot/kernel.old ] ; then chflags -R noschg
> /boot/kernel.old ; rm -rf /boot/kernel.old ; fi ; mv /boot/kernel
> /boot/kernel.old ; if [ -n "/usr/lib/debug" -a -d
> /usr/lib/debug/boot/kernel ]; then rm -rf
> /usr/lib/debug/boot/kernel.old ; mv /usr/lib/debug/boot/kernel
> /usr/lib/debug/boot/kernel.old ; fi ; sysctl
> kern.bootfile=/boot/kernel.old/"`basename "$thiskernel"`" ; fi
> kern.bootfile: /boot/kernel/kernel -> /boot/kernel.old/kernel
> mkdir -p /boot/kernel
> install -p -m 555 -o root -g wheel kernel /boot/kernel/
> mkdir -p /usr/lib/debug/boot/kernel
> install -p -m 555 -o root -g wheel kernel.debug /usr/lib/debug/boot/kernel/
> cd /usr/src/sys/modules;
> MAKEOBJDIRPREFIX=/usr/obj/usr/src/amd64.amd64/sys/vtnet/modules
> KMODDIR=/boot/kernel MACHINE_CPUARCH=amd64 MACHINE=amd64
> MACHINE_ARCH=amd64 MODULES_EXTRA="" WITHOUT_MODULES="" ARCH_FLAGS=""
> DEBUG_FLAGS="-g" __MPATH=""
> KERNBUILDDIR="/usr/obj/usr/src/amd64.amd64/sys/vtnet"
> SYSDIR="/usr/src/sys" MODULE_TIED=yes WITH_CTF="1" KCSAN_ENABLED="yes"
> COMPAT_FREEBSD32_ENABLED="yes" make install
> make[3] warning:
> /usr/obj/usr/src/amd64.amd64/sys/vtnet/modules/usr/src/sys/modules:
> Permission denied.
> ===> aac (install)
> make[4] warning:
> /usr/obj/usr/src/amd64.amd64/sys/vtnet/modules/usr/src/sys/modules/aac:
> Permission denied.
> install -T release -o root -g wheel -m 555 aac.ko /boot/kernel/
> install -T dbg -o root -g wheel -m 555 aac.ko.debug
> /usr/lib/debug/boot/kernel/
> ===> aacraid (install)
> make[4] warning:
> /usr/obj/usr/src/amd64.amd64/sys/vtnet/modules/usr/src/sys/modules/aacraid:
>
> Permission denied.
> install -T release -o root -g wheel -m 555 aacraid.ko /boot/kernel/
> install -T dbg -o root -g wheel -m 555 aacraid.ko.debug
> /usr/lib/debug/boot/kernel/
> ===> accf_data (install)
> make[4] warning:
> /usr/obj/usr/src/amd64.amd64/sys/vtnet/modules/usr/src/sys/modules/accf_data:
>
> Permission denied.
>
>
> eg.
>
> 0{git}% ls -l /usr/lib32/libhei*
> -r--r--r-- 1 root wheel - 53622 Mar 17 20:02 /usr/lib32/libheimbase.a
> lrwxr-xr-x 1 root wheel - 17 May 24 17:11
> /usr/lib32/libheimbase.so -> libheimbase.so.11
> -r--r--r-- 1 root wheel - 13224 May 24 17:11
> /usr/lib32/libheimbase.so.11
> -r--r--r-- 1 root wheel - 54966 Mar 17 20:02 /usr/lib32/libheimbase_p.a
> -r--r--r-- 1 root wheel - 67598 Mar 17 20:02 /usr/lib32/libheimntlm.a
> lrwxr-xr-x 1 root wheel - 17 May 24 17:11
> /usr/lib32/libheimntlm.so -> libheimntlm.so.11
> -r--r--r-- 1 root wheel - 23840 May 24 17:11
> /usr/lib32/libheimntlm.so.11
> -r--r--r-- 1 root wheel - 68746 Mar 17 20:02 /usr/lib32/libheimntlm_p.a
> 0{git}%
>
>
> Looking at the pcap, nothing stands out as to what the problem might be.
> Looking at the replies, the ERROR is just "ERROR: No such file or
> directory" and doesnt correlate to the permission denied errors
>
> reading from file nfs.pcap, link-type EN10MB (Ethernet)
> 2 xid reply ok
> 3371 xid reply ok getattr CHR 11734062556 ids
> 951 xid reply ok getattr CHR 12231260544 ids
> 626 xid reply ok getattr ERROR: No such
> 2 xid reply ok getattr FIFO 10432271160 ids
> 6758 xid reply ok getattr FIFO 10731272141 ids
> 165 xid reply ok getattr FIFO 11433667553 ids
> 214 xid reply ok getattr FIFO 12231260544 ids
> 3373 xid reply ok getattr LNK 10333067563 ids
> 2 xid reply ok getattr LNK 12231267145 ids
> 2 xid reply ok getattr unk-ft 10 10536061550
> 2 xid reply ok getattr unk-ft 13 10334462541
> 2 xid reply ok getattr unk-ft 15 12231261554
> 673 xid reply ok getattr unk-ft 6 10130661545
> 879 xid reply ok getattr unk-ft 6 11433667553
> 126 xid reply ok getattr unk-ft 8 12231260544
> 2 xid reply ok null
>
>
>
>
[-- Attachment #2 --]
<div dir="auto">Sorry for the top post...<div dir="auto"><br></div><div dir="auto">It should work. It has worked in the past. This is a regression. </div><div dir="auto"><br></div><div dir="auto">Warner </div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, May 24, 2022, 12:59 PM mike tancsa <<a href="mailto:mike@sentex.net">mike@sentex.net</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On RELENG_12, I was able to mount via NFS /usr/obj and src and do an <br>
installworld on read only mounts. However, with RELENG_13 and above, I <br>
get permission denied errors. The install seems to continue just fine, <br>
but I am not sure if something subtle is being missed. Looking at what <br>
gets installed, the kernel modules get installed just fine. However, it <br>
seems installworld has some problems with static libs (.a files). They <br>
dont seem to get re-installed. However, not sure if thats by design as <br>
doing a quick checksum, there is no difference on the build server and <br>
target of the ~ 700 .a files I checked in /usr/lib32 and /usr/lib. So <br>
despite the time stamps being off, maybe it just does not bother to <br>
update the target if they have not changed?<br>
<br>
e.g.<br>
<br>
make KERNCONF=vtnet installkernel<br>
<br>
<br>
make warning: /usr/obj/usr/src: Permission denied.<br>
make[1] warning: /usr/src/: Permission denied.<br>
make[2] warning: /usr/obj/usr/src/amd64.amd64: Permission denied.<br>
--------------------------------------------------------------<br>
>>> Install check kernel<br>
--------------------------------------------------------------<br>
--------------------------------------------------------------<br>
>>> Installing kernel vtnet on Tue May 24 13:50:52 EDT 2022<br>
--------------------------------------------------------------<br>
cd /usr/obj/usr/src/amd64.amd64/sys/vtnet; MACHINE_ARCH=amd64 <br>
MACHINE=amd64 CPUTYPE= CC="cc -target x86_64-unknown-freebsd13.1 <br>
--sysroot=/usr/obj/usr/src/amd64.amd64/tmp <br>
-B/usr/obj/usr/src/amd64.amd64/tmp/usr/bin" CXX="c++ -target <br>
x86_64-unknown-freebsd13.1 --sysroot=/usr/obj/usr/src/amd64.amd64/tmp <br>
-B/usr/obj/usr/src/amd64.amd64/tmp/usr/bin" CPP="cpp -target <br>
x86_64-unknown-freebsd13.1 --sysroot=/usr/obj/usr/src/amd64.amd64/tmp <br>
-B/usr/obj/usr/src/amd64.amd64/tmp/usr/bin" AS="as" AR="ar" <br>
ELFCTL="elfctl" LD="ld" LLVM_LINK="" NM=nm OBJCOPY="objcopy" <br>
RANLIB=ranlib STRINGS= SIZE="size" STRIPBIN="strip" <br>
PATH=/usr/obj/usr/src/amd64.amd64/tmp/bin:/usr/obj/usr/src/amd64.amd64/tmp/usr/sbin:/usr/obj/usr/src/amd64.amd64/tmp/usr/bin:/usr/obj/usr/src/amd64.amd64/tmp/legacy/usr/sbin:/usr/obj/usr/src/amd64.amd64/tmp/legacy/usr/bin:/usr/obj/usr/src/amd64.amd64/tmp/legacy/bin:/usr/obj/usr/src/amd64.amd64/tmp/legacy/usr/libexec::/sbin:/bin:/usr/sbin:/usr/bin <br>
make KERNEL=kernel install<br>
make[2] warning: /usr/obj/usr/src/amd64.amd64/sys/vtnet: Permission denied.<br>
thiskernel=`sysctl -n kern.bootfile || echo /boot/kernel/kernel` ; if [ <br>
! "`dirname "$thiskernel"`" -ef /boot/kernel ] ; then chflags -R noschg <br>
/boot/kernel ; rm -rf /boot/kernel ; rm -rf /usr/lib/debug/boot/kernel <br>
; else if [ -d /boot/kernel.old ] ; then chflags -R noschg <br>
/boot/kernel.old ; rm -rf /boot/kernel.old ; fi ; mv /boot/kernel <br>
/boot/kernel.old ; if [ -n "/usr/lib/debug" -a -d <br>
/usr/lib/debug/boot/kernel ]; then rm -rf <br>
/usr/lib/debug/boot/kernel.old ; mv /usr/lib/debug/boot/kernel <br>
/usr/lib/debug/boot/kernel.old ; fi ; sysctl <br>
kern.bootfile=/boot/kernel.old/"`basename "$thiskernel"`" ; fi<br>
kern.bootfile: /boot/kernel/kernel -> /boot/kernel.old/kernel<br>
mkdir -p /boot/kernel<br>
install -p -m 555 -o root -g wheel kernel /boot/kernel/<br>
mkdir -p /usr/lib/debug/boot/kernel<br>
install -p -m 555 -o root -g wheel kernel.debug /usr/lib/debug/boot/kernel/<br>
cd /usr/src/sys/modules; <br>
MAKEOBJDIRPREFIX=/usr/obj/usr/src/amd64.amd64/sys/vtnet/modules <br>
KMODDIR=/boot/kernel MACHINE_CPUARCH=amd64 MACHINE=amd64 <br>
MACHINE_ARCH=amd64 MODULES_EXTRA="" WITHOUT_MODULES="" ARCH_FLAGS="" <br>
DEBUG_FLAGS="-g" __MPATH="" <br>
KERNBUILDDIR="/usr/obj/usr/src/amd64.amd64/sys/vtnet" <br>
SYSDIR="/usr/src/sys" MODULE_TIED=yes WITH_CTF="1" KCSAN_ENABLED="yes" <br>
COMPAT_FREEBSD32_ENABLED="yes" make install<br>
make[3] warning: <br>
/usr/obj/usr/src/amd64.amd64/sys/vtnet/modules/usr/src/sys/modules: <br>
Permission denied.<br>
===> aac (install)<br>
make[4] warning: <br>
/usr/obj/usr/src/amd64.amd64/sys/vtnet/modules/usr/src/sys/modules/aac: <br>
Permission denied.<br>
install -T release -o root -g wheel -m 555 aac.ko /boot/kernel/<br>
install -T dbg -o root -g wheel -m 555 aac.ko.debug <br>
/usr/lib/debug/boot/kernel/<br>
===> aacraid (install)<br>
make[4] warning: <br>
/usr/obj/usr/src/amd64.amd64/sys/vtnet/modules/usr/src/sys/modules/aacraid: <br>
Permission denied.<br>
install -T release -o root -g wheel -m 555 aacraid.ko /boot/kernel/<br>
install -T dbg -o root -g wheel -m 555 aacraid.ko.debug <br>
/usr/lib/debug/boot/kernel/<br>
===> accf_data (install)<br>
make[4] warning: <br>
/usr/obj/usr/src/amd64.amd64/sys/vtnet/modules/usr/src/sys/modules/accf_data: <br>
Permission denied.<br>
<br>
<br>
eg.<br>
<br>
0{git}% ls -l /usr/lib32/libhei*<br>
-r--r--r-- 1 root wheel - 53622 Mar 17 20:02 /usr/lib32/libheimbase.a<br>
lrwxr-xr-x 1 root wheel - 17 May 24 17:11 <br>
/usr/lib32/libheimbase.so -> libheimbase.so.11<br>
-r--r--r-- 1 root wheel - 13224 May 24 17:11 /usr/lib32/libheimbase.so.11<br>
-r--r--r-- 1 root wheel - 54966 Mar 17 20:02 /usr/lib32/libheimbase_p.a<br>
-r--r--r-- 1 root wheel - 67598 Mar 17 20:02 /usr/lib32/libheimntlm.a<br>
lrwxr-xr-x 1 root wheel - 17 May 24 17:11 <br>
/usr/lib32/libheimntlm.so -> libheimntlm.so.11<br>
-r--r--r-- 1 root wheel - 23840 May 24 17:11 /usr/lib32/libheimntlm.so.11<br>
-r--r--r-- 1 root wheel - 68746 Mar 17 20:02 /usr/lib32/libheimntlm_p.a<br>
0{git}%<br>
<br>
<br>
Looking at the pcap, nothing stands out as to what the problem might be. <br>
Looking at the replies, the ERROR is just "ERROR: No such file or <br>
directory" and doesnt correlate to the permission denied errors<br>
<br>
reading from file nfs.pcap, link-type EN10MB (Ethernet)<br>
2 xid reply ok<br>
3371 xid reply ok getattr CHR 11734062556 ids<br>
951 xid reply ok getattr CHR 12231260544 ids<br>
626 xid reply ok getattr ERROR: No such<br>
2 xid reply ok getattr FIFO 10432271160 ids<br>
6758 xid reply ok getattr FIFO 10731272141 ids<br>
165 xid reply ok getattr FIFO 11433667553 ids<br>
214 xid reply ok getattr FIFO 12231260544 ids<br>
3373 xid reply ok getattr LNK 10333067563 ids<br>
2 xid reply ok getattr LNK 12231267145 ids<br>
2 xid reply ok getattr unk-ft 10 10536061550<br>
2 xid reply ok getattr unk-ft 13 10334462541<br>
2 xid reply ok getattr unk-ft 15 12231261554<br>
673 xid reply ok getattr unk-ft 6 10130661545<br>
879 xid reply ok getattr unk-ft 6 11433667553<br>
126 xid reply ok getattr unk-ft 8 12231260544<br>
2 xid reply ok null<br>
<br>
<br>
<br>
</blockquote></div>
help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CANCZdfpF_keGwXAzFsv%2B5GwpBcmYfHDUbai--juaydLQp3RGXg>