Date: Mon, 28 Dec 2020 10:56:22 -0800 From: bob prohaska <fbsd@www.zefox.net> To: Mark Millard <marklmi@yahoo.com> Cc: freebsd-arm <freebsd-arm@freebsd.org>, bob prohaska <fbsd@www.zefox.net> Subject: Re: Migrating from -current to stable/12 on RPI2B (ARMv7) Message-ID: <20201228185622.GB28380@www.zefox.net> In-Reply-To: <F9CB3890-5E07-46C9-AC40-D968F8B51B1F@yahoo.com> References: <20201228044840.GA28380@www.zefox.net> <F9CB3890-5E07-46C9-AC40-D968F8B51B1F@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sun, Dec 27, 2020 at 10:10:18PM -0800, Mark Millard wrote: >=20 >=20 > On 2020-Dec-27, at 20:48, bob prohaska <fbsd@www.zefox.net> wrote: >=20 > > Having a bit of trouble migrating to stable/12 from 13.0-CURRENT=20 > > FreeBSD 13.0-CURRENT #3 r368820: Sat Dec 26 19:01:50 PST 2020 =20 > > bob@www.zefox.com:/usr/obj/usr/src/arm.armv7/sys/GENERIC-MMCCAM arm > >=20 > > This is on a Pi2B v1.1 The MMCCAM kernel option was added for sake=20 > > of the experiment and hasn't, up to now, been accompanied by any > > visible problems. World and kernel have been built many times, no > > problems.=20 > >=20 > > Buildworld keeps stopping at > > --- clang.full --- > > c++ -O -pipe -fno-common -mlong-calls -I/usr/obj/usr/freebsd-src/arm.ar= mv7/tmp/obj-tools/lib/clang/libclang -I/usr/obj/usr/freebsd-src/arm.armv7/t= mp/obj-tools/lib/clang/libllvm -I/usr/freebsd-src/contrib/llvm-project/clan= g/include -I/usr/freebsd-src/lib/clang/include -I/usr/freebsd-src/contrib/l= lvm-project/llvm/include -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D= __STDC_LIMIT_MACROS -DHAVE_VCS_VERSION_INC -DNDEBUG -DLLVM_DEFAULT_TARGET_T= RIPLE=3D\"armv7-unknown-freebsd12.2-gnueabihf\" -DLLVM_HOST_TRIPLE=3D\"armv= 7-unknown-freebsd12.2\" -DDEFAULT_SYSROOT=3D\"/usr/obj/usr/freebsd-src/arm.= armv7/tmp\" -DLLVM_TARGET_ENABLE_ARM -DLLVM_NATIVE_ASMPARSER=3DLLVMInitiali= zeARMAsmParser -DLLVM_NATIVE_ASMPRINTER=3DLLVMInitializeARMAsmPrinter -DLLV= M_NATIVE_DISASSEMBLER=3DLLVMInitializeARMDisassembler -DLLVM_NATIVE_TARGET= =3DLLVMInitializeARMTarget -DLLVM_NATIVE_TARGETINFO=3DLLVMInitializeARMTarg= etInfo -DLLVM_NATIVE_TARGETMC=3DLLVMInitializeARMTargetMC -ffunction-sectio= ns -fdata-sections -gline-tables-only -Wno-for > > mat-zero-length -Qunused-arguments -I/usr/obj/usr/freebsd-src/arm.armv7= /tmp/legacy/usr/include -fno-exceptions -fno-rtti -std=3Dc++14 -stdlib=3Dli= bc++ -Wno-c++11-extensions -Wl,--gc-sections -static -L/usr/obj/usr/freeb= sd-src/arm.armv7/tmp/legacy/usr/lib -o clang.full cc1_main.o cc1as_main.o = cc1gen_reproducer_main.o driver.o /usr/obj/usr/freebsd-src/arm.armv7/tmp/ob= j-tools/lib/clang/libclang/libclang.a /usr/obj/usr/freebsd-src/arm.armv7/tm= p/obj-tools/lib/clang/libllvm/libllvm.a -L/usr/obj/usr/freebsd-src/arm.armv= 7/tmp/obj-tools/lib/libz -lz -L/usr/obj/usr/freebsd-src/arm.armv7/tmp/obj-t= ools/lib/libexecinfo -lexecinfo -L/usr/obj/usr/freebsd-src/arm.armv7/tmp/ob= j-tools/lib/libelf -lelf -L/usr/obj/usr/freebsd-src/arm.armv7/tmp/obj-tools= /lib/ncurses/ncursesw -lncursesw -L/usr/obj/usr/freebsd-src/arm.armv7/tmp/o= bj-tools/lib/libthr -lpthread -legacy > > ld: error: failed to open clang.full: Cannot allocate memory > > c++: error: linker command failed with exit code 1 (use -v to see invoc= ation) > > *** [clang.full] Error code 1 > >=20 > > make[4]: stopped in /usr/freebsd-src/usr.bin/clang/clang > > 1 error > >=20 > > The failure occurs many minutes after the log entry appears.=20 > > No errors on the console, swap tops near half a gig. The=20 > > stable/12 sources being compiled were obtained via git. The=20 > > -current sources used to compile the running system were=20 > > obtained using svnlite.=20 > >=20 > > I don't recall seeing this much swap used before by armv7,=20 > > but that's the only notable oddity. =20 > >=20 > > The stable/12 sources have been updated every day for the=20 > > past few in hopes the trouble might go away, the -current=20 > > sources seem to be updated as far as svnlite goes.=20 > >=20 > > Any suggestions appreciated, thanks for reading > >=20 >=20 > I presume the historical system tuning for small memory systems > from our past investigations and help that we got. >=20 Correct. /boot/loader.conf contains=20 vm.pageout_oom_seq=3D"4096" vm.pfault_oom_attempts=3D"-1" > You do not mention -j4 vs. -j1 or the like. I'm assuming -j1 has > been tried. >=20 j values of 4, 2 and 1 have been tried, to no effect > You have not said if you have adjusted the likes of: > kern.maxswapzone or VM_SWZONE_SIZE_MAX or if you get the boot > notices about the likes of: >=20 > warning: total configured swap (??? pages) exceeds maximum recommended am= ount (??? pages). >=20 > Mistuning these could lead to RAM fragmentation issues as I > understand. > Dmesg reports=20 warning: total configured swap (579552 pages) exceeds maximum recomm ended amount (467936 pages). warning: increase kern.maxswzone or reduce amount of swap. but that hasn't caused recognizable trouble up to now. Adding even more swap doesn't make things any worse.=20 =20 > Since the allocation failure is reported by ld while it tries to > produce clang.full , I wonder if you have used: >=20 > LDFLAGS.lld+=3D -Wl,--no-threads >=20 > in, say, /etc/make.conf ? If not, the llvm ld will try to use > all 4 RPi2 v1.1 cores (via threads, not via processes) and, from > what I've seen, will end up trying to use more RAM than when it > runs single-threaded (main thread only). Of course, links then > take longer to complete. (LLVM ld actually ends up up with 4+2 > threads, counting the processes's main thread as well.) >=20 > Except on the machines with plenty of RAM (by a large margin for > the hardware-thread count), I use --no-threads for contexts where > I can readily control such. >=20 > (Ports do not use LDFLAGS.lld and gnu's linker does not tolerate > --no-threads as a command line option. So "readily" is limited to > buildkernel and buildworld .) > I didn't know about LDFLAGS, but a re-try with=20 -j1 and LDFLAGS.lld+=3D -Wl,--threads=3D1 (apparently the syntax changed) in /etc/make.conf=20 promptly reproduced the error.=20 At this point the host has two source directories: /usr/src, obtained using svnlite, holding -current and owned by root, along with=20 /usr/freebsd-src, obtained using git, holding -stable/12=20 owned by bob (member of wheel) where the error is occuring. Could this be either a permissions problem or a conflict among intermediate files left over between make attempts? Each source directory is matched by its own object directory, so that seems unlikely. There have been no warnings on the=20 console related to memory or swap, only the cryptic little=20 "cannot allocate memory" in the buildworld log. It's especially puzzling that the problem is evident only when compiling stable/12 as a regular user, but not when compiling -current as root. I'm trying a sanity check now by deleting clang.full from /usr/obj/usr/src (two copies) and restarting buildword with -j4 and no LDFLAGS to verify the error isn't present in -current. That might take a day or two.=20 Many thanks for readying and replying! bob prohaska
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20201228185622.GB28380>