Date: Mon, 28 Dec 2020 12:07:36 -0800 From: Mark Millard <marklmi@yahoo.com> To: bob prohaska <fbsd@www.zefox.net> Cc: freebsd-arm <freebsd-arm@freebsd.org> Subject: Re: Migrating from -current to stable/12 on RPI2B (ARMv7) Message-ID: <E1EC1332-62D8-4E51-BF4D-9812AF7EF44B@yahoo.com> In-Reply-To: <20201228185622.GB28380@www.zefox.net> References: <20201228044840.GA28380@www.zefox.net> <F9CB3890-5E07-46C9-AC40-D968F8B51B1F@yahoo.com> <20201228185622.GB28380@www.zefox.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2020-Dec-28, at 10:56, bob prohaska <fbsd at www.zefox.net> wrote: > On Sun, Dec 27, 2020 at 10:10:18PM -0800, Mark Millard wrote: >>=20 >>=20 >> On 2020-Dec-27, at 20:48, bob prohaska <fbsd@www.zefox.net> wrote: >>=20 >>> Having a bit of trouble migrating to stable/12 from 13.0-CURRENT=20 >>> FreeBSD 13.0-CURRENT #3 r368820: Sat Dec 26 19:01:50 PST 2020 =20 >>> bob@www.zefox.com:/usr/obj/usr/src/arm.armv7/sys/GENERIC-MMCCAM arm >>>=20 >>> This is on a Pi2B v1.1 The MMCCAM kernel option was added for sake=20= >>> of the experiment and hasn't, up to now, been accompanied by any >>> visible problems. World and kernel have been built many times, no >>> problems.=20 >>>=20 >>> Buildworld keeps stopping at >>> --- clang.full --- >>> c++ -O -pipe -fno-common -mlong-calls = -I/usr/obj/usr/freebsd-src/arm.armv7/tmp/obj-tools/lib/clang/libclang = -I/usr/obj/usr/freebsd-src/arm.armv7/tmp/obj-tools/lib/clang/libllvm = -I/usr/freebsd-src/contrib/llvm-project/clang/include = -I/usr/freebsd-src/lib/clang/include = -I/usr/freebsd-src/contrib/llvm-project/llvm/include = -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS = -DHAVE_VCS_VERSION_INC -DNDEBUG = -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"armv7-unknown-freebsd12.2-gnueabihf\" = -DLLVM_HOST_TRIPLE=3D\"armv7-unknown-freebsd12.2\" = -DDEFAULT_SYSROOT=3D\"/usr/obj/usr/freebsd-src/arm.armv7/tmp\" = -DLLVM_TARGET_ENABLE_ARM = -DLLVM_NATIVE_ASMPARSER=3DLLVMInitializeARMAsmParser = -DLLVM_NATIVE_ASMPRINTER=3DLLVMInitializeARMAsmPrinter = -DLLVM_NATIVE_DISASSEMBLER=3DLLVMInitializeARMDisassembler = -DLLVM_NATIVE_TARGET=3DLLVMInitializeARMTarget = -DLLVM_NATIVE_TARGETINFO=3DLLVMInitializeARMTargetInfo = -DLLVM_NATIVE_TARGETMC=3DLLVMInitializeARMTargetMC -ffunction-sections = -fdata-sections -gline-tables-only -Wno-for >>> mat-zero-length -Qunused-arguments = -I/usr/obj/usr/freebsd-src/arm.armv7/tmp/legacy/usr/include = -fno-exceptions -fno-rtti -std=3Dc++14 -stdlib=3Dlibc++ = -Wno-c++11-extensions -Wl,--gc-sections -static = -L/usr/obj/usr/freebsd-src/arm.armv7/tmp/legacy/usr/lib -o clang.full = cc1_main.o cc1as_main.o cc1gen_reproducer_main.o driver.o = /usr/obj/usr/freebsd-src/arm.armv7/tmp/obj-tools/lib/clang/libclang/libcla= ng.a = /usr/obj/usr/freebsd-src/arm.armv7/tmp/obj-tools/lib/clang/libllvm/libllvm= .a -L/usr/obj/usr/freebsd-src/arm.armv7/tmp/obj-tools/lib/libz -lz = -L/usr/obj/usr/freebsd-src/arm.armv7/tmp/obj-tools/lib/libexecinfo = -lexecinfo -L/usr/obj/usr/freebsd-src/arm.armv7/tmp/obj-tools/lib/libelf = -lelf = -L/usr/obj/usr/freebsd-src/arm.armv7/tmp/obj-tools/lib/ncurses/ncursesw = -lncursesw -L/usr/obj/usr/freebsd-src/arm.armv7/tmp/obj-tools/lib/libthr = -lpthread -legacy >>> ld: error: failed to open clang.full: Cannot allocate memory >>> c++: error: linker command failed with exit code 1 (use -v to see = invocation) >>> *** [clang.full] Error code 1 >>>=20 >>> make[4]: stopped in /usr/freebsd-src/usr.bin/clang/clang >>> 1 error >>>=20 >>> The failure occurs many minutes after the log entry appears.=20 >>> No errors on the console, swap tops near half a gig. The=20 >>> stable/12 sources being compiled were obtained via git. The=20 >>> -current sources used to compile the running system were=20 >>> obtained using svnlite.=20 >>>=20 >>> I don't recall seeing this much swap used before by armv7,=20 >>> but that's the only notable oddity. =20 >>>=20 >>> The stable/12 sources have been updated every day for the=20 >>> past few in hopes the trouble might go away, the -current=20 >>> sources seem to be updated as far as svnlite goes.=20 >>>=20 >>> Any suggestions appreciated, thanks for reading >>>=20 >>=20 >> I presume the historical system tuning for small memory systems >> from our past investigations and help that we got. >>=20 > Correct. /boot/loader.conf contains=20 > vm.pageout_oom_seq=3D"4096" > vm.pfault_oom_attempts=3D"-1" >=20 >=20 >> You do not mention -j4 vs. -j1 or the like. I'm assuming -j1 has >> been tried. >>=20 > j values of 4, 2 and 1 have been tried, to no effect >=20 >> You have not said if you have adjusted the likes of: >> kern.maxswapzone or VM_SWZONE_SIZE_MAX or if you get the boot >> notices about the likes of: >>=20 >> warning: total configured swap (??? pages) exceeds maximum = recommended amount (??? pages). >>=20 >> Mistuning these could lead to RAM fragmentation issues as I >> understand. >>=20 > Dmesg reports=20 > warning: total configured swap (579552 pages) exceeds maximum recomm > ended amount (467936 pages). > warning: increase kern.maxswzone or reduce amount of swap. > but that hasn't caused recognizable trouble up to now. Adding > even more swap doesn't make things any worse.=20 The question is more if having somewhat less than 467936 pages of swap leads to less problems with allocating possibly sizable contiguous RAM if such is being attempted. (It may not help.) >> Since the allocation failure is reported by ld while it tries to >> produce clang.full , I wonder if you have used: >>=20 >> LDFLAGS.lld+=3D -Wl,--no-threads >>=20 >> in, say, /etc/make.conf ? If not, the llvm ld will try to use >> all 4 RPi2 v1.1 cores (via threads, not via processes) and, from >> what I've seen, will end up trying to use more RAM than when it >> runs single-threaded (main thread only). Of course, links then >> take longer to complete. (LLVM ld actually ends up up with 4+2 >> threads, counting the processes's main thread as well.) >>=20 >> Except on the machines with plenty of RAM (by a large margin for >> the hardware-thread count), I use --no-threads for contexts where >> I can readily control such. >>=20 >> (Ports do not use LDFLAGS.lld and gnu's linker does not tolerate >> --no-threads as a command line option. So "readily" is limited to >> buildkernel and buildworld .) >>=20 > I didn't know about LDFLAGS, but a re-try with=20 > -j1 and LDFLAGS.lld+=3D -Wl,--threads=3D1 > (apparently the syntax changed) in /etc/make.conf=20 > promptly reproduced the error.=20 Hmm. It been a while since I did a native build instead of a cross build. The cross build context has RAM and does not use the assignment so I'd not noticed. Thanks for the report! > At this point the host has two source directories: > /usr/src, obtained using svnlite, holding -current > and owned by root, along with=20 > /usr/freebsd-src, obtained using git, holding -stable/12=20 > owned by bob (member of wheel) where the error is occuring. >=20 > Could this be either a permissions problem or a conflict > among intermediate files left over between make attempts? > Each source directory is matched by its own object directory, > so that seems unlikely. There have been no warnings on the=20 > console related to memory or swap, only the cryptic little=20 > "cannot allocate memory" in the buildworld log. >=20 > It's especially puzzling that the problem is evident only > when compiling stable/12 as a regular user, but not when > compiling -current as root. I've not experimented with regular-user based builds at all, just root based builds with the files own by root. Have you done regular-user based builds before? Is this new behavior for something that used to to work? > I'm trying a sanity check now by deleting clang.full from > /usr/obj/usr/src (two copies) and restarting buildword with > -j4 and no LDFLAGS to verify the error isn't present in > -current. That might take a day or two.=20 =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?E1EC1332-62D8-4E51-BF4D-9812AF7EF44B>