From owner-freebsd-hackers@freebsd.org Sat Mar 13 20:43:16 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3F5605AAF9B for ; Sat, 13 Mar 2021 20:43:16 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DyZQD1Fpdz3P2W; Sat, 13 Mar 2021 20:43:16 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from mousie.catspoiler.org (unknown [76.212.85.177]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) (Authenticated sender: truckman) by smtp.freebsd.org (Postfix) with ESMTPSA id 929FA6EF9; Sat, 13 Mar 2021 20:43:15 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Date: Sat, 13 Mar 2021 12:43:13 -0800 (PST) From: Don Lewis Subject: Re: problems building a release To: Mark Millard cc: freebsd-hackers In-Reply-To: <8CE18E0A-8931-4C41-83B4-D30EEC049E6D@yahoo.com> Message-ID: References: <8CE18E0A-8931-4C41-83B4-D30EEC049E6D@yahoo.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; CHARSET=iso-8859-13 Content-Transfer-Encoding: QUOTED-PRINTABLE Content-Disposition: INLINE X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Mar 2021 20:43:16 -0000 On 13 Mar, Mark Millard wrote: > On 2021-Mar-13, at 00:11, Don Lewis wrote: >=20 >> On 10 Mar, Mark Millard wrote: >>> On 2021-Mar-10, at 15:00, Don Lewis wrote: >>>=20 >>>> I'm trying to build a customized 13.0 i386 release on a amd64 host >>>> running FreeBSD 14.0-CURRENT #89 n245058-1ca8842f3ad9. The source tre= e >>>> is branch releng/13.0 from last night. The release build gets most of >>>> the way through, but fails with: >>>>=20 >>>> =3D=3D=3D> usr.bin/svn/lib/libsvn_wc (installconfig) >>>> --- installconfig_subdir_usr.sbin --- >>>> --- installconfig_subdir_usr.sbin/pkg --- >>>> --- installdirs-CONFSDIR --- >>>> --- _CONFSINS_FreeBSD.conf.quarterly --- >>>> --- installdirs-CONFSDIR --- >>>> installing DIRS CONFSDIR >>>> install -N /scratch/usr/src/etc -d -m 0755 -o root -g wheel /scratc= h/etc/pkg >>>> --- _CONFSINS_FreeBSD.conf.quarterly --- >>>> install -N /scratch/usr/src/etc -C -o root -g wheel -m 644 /scratch= /usr/src/usr.sbin/pkg/FreeBSD.conf.quarterly /scratch/etc/pkg/FreeBSD.conf >>>> Shared object "libedit.so.8" not found, required by "sh" >>>>=20 >>>> /scratch/bin/sh seems to work, though it is using the host library: >>>> # ldd /scratch/bin/sh >>>> /scratch/bin/sh: >>>> =09libedit.so.8 =3D> /usr/lib32/libedit.so.8 (0x20463000) >>>> =09libc.so.7 =3D> /usr/lib32/libc.so.7 (0x20495000) >>>> =09libncursesw.so.9 =3D> /usr/lib32/libncursesw.so.9 (0x20683000) >>>> # /scratch/bin/sh >>>> #=20 >>>>=20 >>>> The library is present in /scratch/lib: >>>> # file /scratch/lib/libedit.so.8 >>>> /scratch/lib/libedit.so.8: ELF 32-bit LSB shared object, Intel 80386, = version 1 (FreeBSD), dynamically linked, for FreeBSD 13.0 (1300139), stripp= ed >>>>=20 >>>> Unfortunately the build isn't verbose enough to give me any hints abou= t >>>> where the problem might be. I've been able to build FreeBSD 11.x and >>>> 12.x releases this way, but no luck so far with 13. >>>>=20 >>>> Update: I am able to build an amd64 release, so the problem has >>>> something to do with crossbuilds. There are some other problems that >>>> crop up, though. >>>>=20 >>>>=20 >>>> =3D=3D=3D> usr.bin/svn/lib/libsvn_wc (installconfig) >>>> --- installconfig_subdir_usr.sbin --- >>>> --- installconfig_subdir_usr.sbin/pkg --- >>>> --- installdirs-CONFSDIR --- >>>> --- _CONFSINS_FreeBSD.conf.quarterly --- >>>> --- installdirs-CONFSDIR --- >>>> installing DIRS CONFSDIR >>>> install -N /scratch/usr/src/etc -d -m 0755 -o root -g wheel /scratc= h/etc/pkg >>>> --- _CONFSINS_FreeBSD.conf.quarterly --- >>>> install -N /scratch/usr/src/etc -C -o root -g wheel -m 644 /scratch= /usr/src/usr.sbin/pkg/FreeBSD.conf.quarterly /scratch/etc/pkg/FreeBSD.conf >>>> ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/l= ocal/lib/perl5/5.20/mach/CORE >>>> 32-bit compatibility ldconfig path: /usr/lib32 >>>> pkg: Warning: Major OS version upgrade detected. Running "pkg-static = install -f pkg" recommended >>>> Updating FreeBSD repository catalogue... >>>> Fetching meta.txz: 100% 916 B 0.9kB/s 00:01 =20 >>>> pkg: repository meta /var/db/pkg/FreeBSD.meta has wrong version 2 >>>> repository FreeBSD has no meta file, using default settings >>>> Fetching packagesite.txz: 100% 6 MiB 1.6MB/s 00:04 =20 >>>> pkg: repository meta /var/db/pkg/FreeBSD.meta has wrong version 2 >>>> Unable to open created repository FreeBSD >>>> Unable to update repository FreeBSD >>>> All repositories are up-to-date. >>>> pkg: repository meta /var/db/pkg/FreeBSD.meta has wrong version 2 >>>> pkg: Repository FreeBSD cannot be opened. 'pkg update' required >>>> Updating database digests format: 100% >>>> pkg: No packages available to install matching 'devel/git' have been f= ound in the repositories >>>=20 >>>=20 >>> To my knowledge FreeBSD does not support 14 directly >>> building 13 or before: self hosted and sufficiently >>> recent older building newer only, as far as direct >>> builds go. >>=20 >> That doesn't seem to be the problem that I'm running into. I run into a >> similar problem when trying to build a 14-CURRENT release. The cause of >> the problem seems to be that I'm specifying TARGET=3Di386 and TARGET_ARC= H=3Di386 >> in both release.conf, and the make.conf that release.conf points to. >> With 14-CURRENT the failure mode is that something can't find ld-elf.so >> and aborts. That happens a number of times during the release build, >> but this time the release build appears to run to completion. If I >> remove the make.conf settings, then the error goes away. The reason >> that I did this in the first place was so I could set TARGET_CPUTYPE. >> This setting appears to be ignored in release.conf. If I set it in >> make.conf without setting TARGET=3Di386 and TARGET_ARCH=3Di386, then I g= et a >> compiler error because the CPUTYPE is not valid on amd64. Basically, >> the difference is that with the settings in both places, I get an i386 >> chroot under /scratch and with the settings different, the chroot is >> amd64. I thing the failure is being caused when something tries to run >> an amd64 executable in under the i386 chroot. Verbosity is too low to >> see what it is. >>=20 >> Another complication is that I want to maintain the target system with >> freebsd-update, so I need to package up my customized make.conf and >> src.conf into the release so they get used by the update builder. It >> should be easy to tweak the release script to do this, but it gets >> complicated if the the make.conf files have different TARGET settings. >>=20 >=20 > You are definitely working outside my direct > experience. But I'll note that > release/release.conf.sample and the man page > have all of the following, not that I > understand the various implications of use: >=20 > ## Set to override the default target architecture. > #TARGET=3D"amd64" > #TARGET_ARCH=3D"amd64" > #KERNEL=3D"GENERIC" > ## Multiple kernels may be set. > #KERNEL=3D"GENERIC XENHVM" > . . . > ## Set to pass additional flags to make(1) for the build chroot setup, su= ch > ## as TARGET/TARGET_ARCH. > #CHROOT_MAKEENV=3D >=20 > Also described via: >=20 > CHROOT_MAKEENV Additional make(1) arguments to pass through, which > directly affect the tuning of the build chroot. >=20 > TARGET and TARGET_ARCH are also listed as controllable > via the ENVIRONMENT : >=20 > TARGET The target hardware platform. This is analogous to= the > =B4uname -m=A1 output. This is necessary to cross-= build > some target architectures. For example, cross-buil= ding > for ARM64 machines requires TARGET_ARCH=3Daarch64 a= nd > TARGET=3Darm64. If not set, TARGET defaults to the > current hardware platform. >=20 > TARGET_ARCH The target machine processor architecture. This is > analogous to the =B4uname -p=A1 output. Set this t= o cross- > build for a different architecture. If not set, > TARGET_ARCH defaults to the current machine > architecture, unless TARGET is also set, in which c= ase > it defaults to the appropriate value for that platf= orm. > Typically, one only needs to set TARGET. >=20 >=20 > A possibility would seem to be use of: >=20 > ## Set to the hardware platform of the target userland. This value > ## is passed to make(1) to set the TARGET (value of uname -m) to cross > ## build. > #EMBEDDED_TARGET=3D >=20 > ## Set to the machine processor architecture of the target userland. > ## This value is passed to make(1) to set the TARGET_ARCH (value of uname= -p) > ## to cross build. > #EMBEDDED_TARGET_ARCH=3D >=20 > May be EMBEDDEDBUILD=3D with an empty or nonexistent EMBEDDEDPORTS > might happen to do somethings useful for your context? (Implication > of WITH_DVD being unset when EMBEDDEDBUILD=3D is in use. Implicit > EMBEDDEDPORTS such as devel/subversion ?) >=20 >=20 > A different direction is to have release use a pre-existing build > (separately built beforehand). As I vaguely remember, this involved: >=20 > ## Set to skip the chroot environment buildworld/installworld/distributio= n > ## step if it is expected the build environment will exist via alternate > ## means. > #CHROOTBUILD_SKIP=3D >=20 > Also described via: >=20 > CHROOTBUILD_SKIP > If defined, the buildworld, installworld, and > distribution stages of the chroot(8) build environm= ent > setup are skipped. This is intended solely for cas= es > where the chroot(8) userland are provided by altern= ate > means. >=20 >=20 > There are also: >=20 > ## Redefine environment variables here to override prototypes > ## defined in release.sh. > #load_chroot_env() { } > #load_target_env() { } > #buildenv_setup() { } Thanks. All of this stuff is pretty confusing. If I specify MAKE_CONF in release.conf, it does affect the target build. In particular MODULES_OVERRIDE does the right thing. It seems like there should be a way to specify a make.conf and src.conf so that they get included in the release so that if the release is installed and then world and kernel are built it reproduces itself. I did manage to unmute release.sh and make so that the build is verbose, but the error I get when I specify TARGET and TARGET_ARCH in the make.conf is still a mystery. There doesn't seem to be a line buffering problem hiding things, this was captured from the terminal window with no redirections involved. =3D=3D=3D> etc (installconfig) mkdir -p bootonly/usr/freebsd-dist cp MANIFEST bootonly/usr/freebsd-dist ln -fs /tmp/bsdinstall_etc/resolv.conf bootonly/etc/resolv.conf echo sendmail_enable=3D\"NONE\" > bootonly/etc/rc.conf echo hostid_enable=3D\"NO\" >> bootonly/etc/rc.conf echo debug.witness.trace=3D0 >> bootonly/etc/sysctl.conf echo vfs.mountroot.timeout=3D\"10\" >> bootonly/boot/loader.conf echo kernels_autodetect=3D\"NO\" >> bootonly/boot/loader.conf cp /usr/src/release/rc.local bootonly/etc sh /usr/src/release/i386/mkisoimages.sh -b 14_0_CURRENT_i386_BO bootonly.is= o bootonly=20 sh /usr/src/release/i386/make-memstick.sh disc1 memstick.img Calculated size of `memstick.img.part': 624754688 bytes, 13074 inodes Extent size set to 32768 memstick.img.part: 595.8MB (1220224 sectors) block size 32768, fragment siz= e 4096 =09using 1 cylinder groups of 595.81MB, 19066 blks, 13952 inodes. super-block backups (for fsck -b #) at: 192, Populating `memstick.img.part' Image `memstick.img.part' complete sh /usr/src/release/i386/make-memstick.sh bootonly mini-memstick.img Calculated size of `mini-memstick.img.part': 237436928 bytes, 13069 inodes Extent size set to 32768 mini-memstick.img.part: 226.4MB (463744 sectors) block size 32768, fragment= size 4096 =09using 1 cylinder groups of 226.44MB, 7246 blks, 14592 inodes. super-block backups (for fsck -b #) at: 192, Populating `mini-memstick.img.part' Image `mini-memstick.img.part' complete make -C /usr/src/release release-done ELF interpreter /libexec/ld-elf.so.1 not found, error 8 Abort trap ELF interpreter /libexec/ld-elf.so.1 not found, error 8 Abort trap ELF interpreter /libexec/ld-elf.so.1 not found, error 8 Abort trap ELF interpreter /libexec/ld-elf.so.1 not found, error 8 Abort trap ELF interpreter /libexec/ld-elf.so.1 not found, error 8 Abort trap ELF interpreter /libexec/ld-elf.so.1 not found, error 8 Abort trap touch release true + eval chroot /scratch make -C /usr/src/release 'TARGET=3Di386' 'TARGET_ARC= H=3Di386' 'KERNCONF=3D"GW"' '__MAKE_CONF=3D/home/dl/gw-release/make.conf' '= SRCCONF=3D/home/dl/gw-release/src.conf' 'NOPORTS=3Dyes' 'WITH_DVD=3D' 'WITH= _VMIMAGES=3D' 'WITH_CLOUDWARE=3D' 'XZ_THREADS=3D0' install 'DESTDIR=3D/R' '= WITH_COMPRESSED_IMAGES=3D' 'WITH_COMPRESSED_VMIMAGES=3D' + chroot /scratch make -C /usr/src/release 'TARGET=3Di386' 'TARGET_ARCH=3Di= 386' 'KERNCONF=3DGW' '__MAKE_CONF=3D/home/dl/gw-release/make.conf' 'SRCCONF= =3D/home/dl/gw-release/src.conf' 'NOPORTS=3Dyes' 'WITH_DVD=3D' 'WITH_VMIMAG= ES=3D' 'WITH_CLOUDWARE=3D' 'XZ_THREADS=3D0' install 'DESTDIR=3D/R' 'WITH_CO= MPRESSED_IMAGES=3D' 'WITH_COMPRESSED_VMIMAGES=3D' ELF interpreter /libexec/ld-elf.so.1 not found, error 8 Abort trap ELF interpreter /libexec/ld-elf.so.1 not found, error 8 Abort trap ELF interpreter /libexec/ld-elf.so.1 not found, error 8 Abort trap ELF interpreter /libexec/ld-elf.so.1 not found, error 8 Abort trap ELF interpreter /libexec/ld-elf.so.1 not found, error 8 Abort trap ELF interpreter /libexec/ld-elf.so.1 not found, error 8 Abort trap mkdir -p /R cp -a ftp /R/ cp -p disc1.iso /R/FreeBSD-14.0-CURRENT-i386-disc1.iso cp -p bootonly.iso /R/FreeBSD-14.0-CURRENT-i386-bootonly.iso cp -p memstick.img /R/FreeBSD-14.0-CURRENT-i386-memstick.img cp -p mini-memstick.img /R/FreeBSD-14.0-CURRENT-i386-mini-memstick.img cd /R && sha512 FreeBSD-14.0-CURRENT-i386* > /R/CHECKSUM.SHA512 cd /R && sha256 FreeBSD-14.0-CURRENT-i386* > /R/CHECKSUM.SHA256 + return 0 + return 0 + umount /scratch/dev