Date: Mon, 18 Jan 2021 21:12:43 -0800 From: Mark Millard <marklmi@yahoo.com> To: bob prohaska <fbsd@www.zefox.net> Cc: Current FreeBSD <freebsd-current@freebsd.org>, freebsd-arm@freebsd.org Subject: Re: Silent hang in buildworld, was Re: Invoking -v for clang during buildworld Message-ID: <D9878BB6-2693-4A04-9E1C-126E0D378F7B@yahoo.com> In-Reply-To: <00104FAD-E32B-4DDE-80DD-FCEF14CEC06B@yahoo.com> References: <20210116043740.GA19523@www.zefox.net> <ED26508F-282D-439D-8A6A-65A136C76C84@yahoo.com> <20210116155538.GA24259@www.zefox.net> <F62027C8-0813-4E6F-934A-3088F966AF8C@yahoo.com> <20210116220334.GA26756@www.zefox.net> <C75D3D9C-4284-4BE3-B2CD-5DC6BBB60843@yahoo.com> <20210117174006.GA30728@www.zefox.net> <85889EAE-F579-4220-9185-944D9AA5075A@yahoo.com> <20210118015009.GA31353@www.zefox.net> <60CCCDE8-E3D3-4920-9FC0-A945330F6830@yahoo.com> <A0427375-5515-4D3C-AF2A-915E60A836A7@yahoo.com> <00104FAD-E32B-4DDE-80DD-FCEF14CEC06B@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2021-Jan-18, at 19:19, Mark Millard <marklmi at yahoo.com> wrote:
> . . .
>> FYI: I re-established my access to a RPi2B V1.1 and made
>> it report: "maximum recommended amount (468832 pages)"
>>=20
>> (The figure can vary some from release to release.)
>>=20
>> 468832*4096 =3D=3D 1920335872 or a little over 1831 MiBytes
>>=20
>> For the 4096 Byte pages, that means that the following from
>> gpart fits without complaint (size is in blocks, not pages):
>>=20
>> 413140992 3686400 da0p2 freebsd-swap (1.8G)
>>=20
>> 3686400*512 is a little over 1.75 GiByte or 1800 MiByte. So
>> I've left some room below 1831 MiBytes, but not a lot.
>>=20
>> FYI about my build experiment that is running:
>>=20
>> # sysctl hw.physmem
>> hw.physmem: 979042304
>>=20
>> which, in recent times for armv7, I can (and did) set in
>> /boot/loader.conf on a faster cortex-A7 SBC (that can boot
>> the same media but has more RAM).
>>=20
>> So I tried a -j4 build, but with LDFLAGS.lld+=3D -Wl,--threads=3D1
>> in use and my other particular src.conf/make.conf like content
>> (so the builds do likely differ from yours in various ways).
>> My build is producing a non-debug build (but with -g symbols).
>> Somewhat after where your buildworld.log stops, my odd variant
>> of top was reporting:
>>=20
>> Mem: . . . , 753672Ki MaxObsActive, 200412Ki MaxObsWired, 892732Ki =
MaxObs(Act+Wir)
>> Swap: . . . , 145832Ki MaxObsUsed
>>=20
>> and top was also showing lots of processes as having "0B" RES
>> in STATE "wait" or "nanslp" (so, apparently swapped out, not paging).
>> ("MaxObs" is short for "maximum observed".)
>>=20
>> For comparison, your swapscript.log reported a maximum total of
>> 346192 KiBytes "Used" for swap, about 98% into the log file.
>>=20
>> (Time goes by . . .)
>>=20
>> It finished with building libllvm and is part way into building
>> libclang. This is probably well past where your hangup happened,
>> given that your published buildworldlog file stopped with
>> libllvm's Target/ARM/ARMMCInstLower.o . My odd top now shows:
>>=20
>> Mem: . . . , 753672Ki MaxObsActive, 200412Ki MaxObsWired, 892732Ki =
MaxObs(Act+Wir)
>> Swap: . . . , 392328Ki MaxObsUsed
>>=20
>> The build continues to run. I'll let you know how it goes.
>> . . .
>=20
> Just after libclang finished my odd top showed:
>=20
> Mem: . . . , 753672Ki MaxObsActive, 200412Ki MaxObsWired, 892736Ki =
MaxObs(Act+Wir)
> Swap: . . . , 537588Ki MaxObsUsed
>=20
> After liblldb:
>=20
> Mem: . . . , 753672Ki MaxObsActive, 200412Ki MaxObsWired, 899276Ki =
MaxObs(Act+Wir)
> Swap: . . . , 537588Ki MaxObsUsed
>=20
> Much later, after the lldb program had been built:
>=20
> Mem: . . . , 765700Ki MaxObsActive, 200412Ki MaxObsWired, 954116Ki =
MaxObs(Act+Wir)
> Swap: . . . , 537588Ki MaxObsUsed
>=20
>>>> World build completed on Mon Jan 18 19:10:08 PST 2021
>>>> World built in 72960 seconds, ncpu: 4, make -j4
>=20
> This was building from scratch what was already installed:
>=20
> # ~/fbsd-based-on-what-freebsd-main.sh=20
> merge-base: 818390ce0ca539300dd15d7a817784f1e3f7a9b8
> merge-base: CommitDate: 2021-01-13 21:27:44 +0000
> 4180404713ec (HEAD -> mm-src) mm-src snapshot for mm's patched build =
in git context.
> 818390ce0ca5 (freebsd/main, freebsd/HEAD, pure-src, main) arm64: fix =
early devmap assertion
> FreeBSD OPiP2E_RPi2v11 13.0-CURRENT FreeBSD 13.0-CURRENT =
mm-src-c255938-g4180404713ec GENERIC-NODBG arm armv7 1300135 1300135
>=20
> This suggests that you should be able to build on the RPi2B v1.1,
> using -j4, with appropriate configuration for what and how to build.
>=20
>=20
> It is now building the matching kernel, my GENERIC-NODBG style.
Done:
>>> Kernel build for GENERIC-NODBG completed on Mon Jan 18 20:33:26 PST =
2021
>>> Kernel(s) GENERIC-NODBG built in 4998 seconds, ncpu: 4, make -j4
So, World+Kernel in somewhat under 22 hours.
The "MaxObs*" figures were unchanged, so:
Mem: . . . , 765700Ki MaxObsActive, 200412Ki MaxObsWired, 954116Ki =
MaxObs(Act+Wir)
Swap: . . . , 537588Ki MaxObsUsed
This suggests that, for now, 800 MiByte of swap would be something
more than 1.5 times what it actually used and 1050 MiBytes would
be something like 2.0 times what it actually used, so leaving some
notable margin for variations in peek usage, at least when linker
threading is avoided.
As for what I used to control "what and how to build" . . .
# more =
~/sys_build_scripts.armv7-host/make_armv7_nodebug_clang_bootstrap-armv7-ho=
st.sh=20
kldload -n filemon && \
script =
~/sys_typescripts/typescript_make_armv7_nodebug_clang_bootstrap-armv7-host=
-$(date +%Y-%m-%d:%H:%M:%S) \
env __MAKE_CONF=3D"/root/src.configs/make.conf" SRCCONF=3D"/dev/null" =
SRC_ENV_CONF=3D"/root/src.configs/src.conf.armv7-clang-bootstrap.armv7-hos=
t" \
WITH_META_MODE=3Dyes \
WORLD_FLAGS=3D"${WORLD_FLAGS} UBLDR_LOADADDR=3D0x42000000" \
MAKEOBJDIRPREFIX=3D"/usr/obj/armv7_clang/arm.armv7" \
make $*
(In my context, UBLDR_LOADADDR is ignored by anything that
can not use the figure given. So I've no bothered to be
more selective about having it in the armv7 builds.)
# more ~/src.configs/make.conf
LDFLAGS.lld+=3D -Wl,--threads=3D1
# more ~/src.configs/src.conf.armv7-clang-bootstrap.armv7-host
TO_TYPE=3Darmv7
#
KERNCONF=3DGENERIC-NODBG
TARGET=3Darm
.if ${.MAKE.LEVEL} =3D=3D 0
TARGET_ARCH=3D${TO_TYPE}
.export TARGET_ARCH
.endif
#
#WITH_CROSS_COMPILER=3D
WITH_SYSTEM_COMPILER=3D
WITH_SYSTEM_LINKER=3D
#
WITH_LIBCPLUSPLUS=3D
WITHOUT_BINUTILS_BOOTSTRAP=3D
WITH_ELFTOOLCHAIN_BOOTSTRAP=3D
#Disables avoiding bootstrap: WITHOUT_LLVM_TARGET_ALL=3D
WITHOUT_LLVM_TARGET_AARCH64=3D
WITH_LLVM_TARGET_ARM=3D
WITHOUT_LLVM_TARGET_MIPS=3D
WITHOUT_LLVM_TARGET_POWERPC=3D
WITHOUT_LLVM_TARGET_RISCV=3D
WITHOUT_LLVM_TARGET_X86=3D
WITH_CLANG=3D
WITH_CLANG_IS_CC=3D
WITH_CLANG_FULL=3D
WITH_CLANG_EXTRAS=3D
WITH_LLD=3D
WITH_LLD_IS_LD=3D
WITHOUT_BINUTILS=3D
#
WITH_LLDB=3D
#
WITH_BOOT=3D
WITHOUT_LIB32=3D
#
#
WITHOUT_WERROR=3D
#WERROR=3D
MALLOC_PRODUCTION=3D
WITH_MALLOC_PRODUCTION=3D
WITHOUT_ASSERT_DEBUG=3D
WITHOUT_LLVM_ASSERTIONS=3D
#
# Avoid stripping but do not control host -g status as well:
DEBUG_FLAGS+=3D
#
WITH_REPRODUCIBLE_BUILD=3D
WITH_DEBUG_FILES=3D
#
# Use of the .clang 's here avoids
# interfering with other C<?>FLAGS
# usage, such as ?=3D usage.
CFLAGS.clang+=3D -mcpu=3Dcortex-a7
CXXFLAGS.clang+=3D -mcpu=3Dcortex-a7
CPPFLAGS.clang+=3D -mcpu=3Dcortex-a7
(I do not claim that you would want WITH_REPRODUCIBLE_BUILD .
I just happen to have been experimenting with it. You might
not want to be explicit about the cpu to target. You might
not want WITH_CLANG_EXTRAS .)
# more /usr/fbsd/mm-src/sys/arm/conf/GENERIC-NODBG
include "GENERIC"
ident GENERIC-NODBG
makeoptions DEBUG=3D-g # Build kernel with gdb(1) =
debug symbols
options AUDIT # Not enabled by default in =
armv7/v6 kernels
# Enabled here to allow kyua =
test runs to
# possibly report auditing =
works.
options ALT_BREAK_TO_DEBUGGER
options KDB # Enable kernel debugger support
# For minimum debugger support (stable branch) use:
options KDB_TRACE # Print a stack trace for a =
panic
options DDB # Enable the kernel debugger
# Extra stuff:
#options VERBOSE_SYSINIT=3D0 # Enable verbose sysinit =
messages
#options BOOTVERBOSE=3D1
#options BOOTHOWTO=3DRB_VERBOSE
options ALT_BREAK_TO_DEBUGGER # Enter debugger on keyboard =
escape sequence
options KLD_DEBUG
#options KTR
#options KTR_MASK=3DKTR_TRAP
##options KTR_CPUMASK=3D0xF
#options KTR_VERBOSE
# Disable any extra checking for. . .
nooptions INVARIANTS # Enable calls of extra sanity =
checking
nooptions INVARIANT_SUPPORT # Extra sanity checks of =
internal structures, required by INVARIANTS
nooptions WITNESS # Enable checks to detect =
deadlocks and cycles
nooptions WITNESS_SKIPSPIN # Don't run witness on spinlocks =
for speed
nooptions DEADLKRES # Enable the deadlock resolver
nooptions MALLOC_DEBUG_MAXZONES # Separate malloc(9) zones
nooptions DIAGNOSTIC
nooptions BUF_TRACKING
nooptions FULL_BUF_TRACKING
nooptions USB_DEBUG
nooptions USB_REQ_DEBUG
nooptions USB_VERBOSE
The /boot/loader.conf file and the /etc/sysctl.conf files
both contained:
vm.pageout_oom_seq=3D120
vm.pfault_oom_attempts=3D-1
(The hw.physmem=3D979042304 in /boot/loader.conf was very-special,
to better approximate your environment. I also controlled the
cpu frequency used via a line in /etc/sysctl.conf . I do not
bother with such non-default frequency usage [or related settings]
for RPi*'s with the pre-RPi4B style power connections but do
control the frequency for the OPi+2E.)
=3D=3D=3D
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?D9878BB6-2693-4A04-9E1C-126E0D378F7B>