From owner-freebsd-arm@freebsd.org Wed Jan 20 01:42:28 2021 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 349CE4E56DC for ; Wed, 20 Jan 2021 01:42:28 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic315-55.consmr.mail.gq1.yahoo.com (sonic315-55.consmr.mail.gq1.yahoo.com [98.137.65.31]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4DL7Yt2dh6z3H30 for ; Wed, 20 Jan 2021 01:42:25 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1611106944; bh=KFPXpALPxR/GAi5tkYuVizVV/DP6N2LVmO92bloGEu7=; h=Subject:From:Date:To:From:Subject:Reply-To; b=oys/EgIvY06uJ3Pf2JfKLPFTcz7ejFxDwPyaXhGnZAUBT1mPldVZykadHdo+/n1ImA0I0yycXMVTl+omhGAJslQjJjlBRlOm/iBbsJoSi+CE02DPYn47coeuf2dxe0z2C+KsiRPaf4ByCydVzj54kZhfqfXy5RdqzzKVWwq0KFyr3V/6WfOD70sXkL88ypKhFd4PPx62i3hppdGe2tb0/5ys/9MiNHrsP5dryxIPnK/Kumt9QCzIKhaGVvuocAfu0gVvZp29fFT6cOzJFMO879MEYH7YCqNiyZeNTiJEeX4vBMHEgAOIest1B93AYCYQ3zHzguCyCLXgN1a3fT5IMQ== X-YMail-OSG: iHSptXsVM1mVwaFY.D9rVclZJSU8HiWlxzLa57yRvQKdv2CxorzuefYXjnmBMLH LEmApRaNCJUSQ.o1W0IMiadeQbmnDC18kyiTYY3qG1AzoPdcAyYM16f7N.IBSE8DdJMCzkksNooi .zAs.fKL.9Kp3ZGmQVD5y3WUJMylIzkJgkAmZ_pNgFoQF5imI5O.gIAFvBe4CMFdH2WSMrjgFiJY 3ZZQUP1lPWKVA8s7hhSmI.N6L6KNafU0HwPsHRXeqPr3Rl5XA43ZwyW81OW9LEGdbyDAwkHZY9HF aM_NTPWl_wWa70jyAd9CK7HS4JLlmPtIvCs8.Jmwb2ESHPvzI31i4cJLQ4Sw7ffSQvtwN_JPtNIq ocEkGEIUE29BFJil08dNJhcPMAuuS8HV.G5n2fBP4PdqllesJ9ANndn3SPE_OrlcC0DsZnztN_Bu SPxV_JTrwTj.Q1AZToE_XsUDmf7AS3T32pr3OIJ9RZqxOMArEf2kfIjSOzPxunAzt9rhhiG0pjZD qjnqpZX.xxPI7Gjj1fOU9SlXq9jZYwIkv5BejKj6MNn80HKkxfhmMT2Q1tPIiD8UQG58qKPm2mJc bIMEkQ_z8T4gzJImqTq52HkMStQMHBKg_v8Pj42qCLNtSvglp2982495NXcEuJZ7k1SKAIs7lYPQ JhDqL6Gc5ar06v7PO8f2dq92LdoxkcfEckbDLQJ6eqDOnC2yCyW3bua469rhrW8vtdCyfxYWnsph 52WgW2Mm.z7ymf4wnWRY4pfO8D42Lz89nP1kQACzfLW.sh_zNyZlDCr6MKDL8BM3vLD0mvaamwbi REU1T0Xt9eVd6gLQ30xeRNsF0U9PUFz5KzTi.g.1cKLet9Uz6VxjckZ4L1844Wf0nHau0s2OVzf8 uoI4NvKfqPR3IMAO2i4BIZTnqcY2syjowsQKKlhcGSOD1zJ9bA8kp8vkK8WzkUnKt6_1AKZt_QGN mPhwEU6hBzY43zCe31PWigJVhQouw1arJXaMfPsvfL1NEa1m8KyGyJKbiJ1JP6nDVXmKfPvgiSeA oha9RF85YntQWVDSb9vBmT34UXeScn2CEHgpI7Zi96cQOzXY4q7RNHRAlrBCvqP9afITL4mKAXSP PJ3weSRzUJ0xsd.XKuVnc6UJ9eIT6Zw6WdJ0AIP_T7gVniYSsh1nOXA5W.5C0ZA9hBQCieayiWja JhoTLAjtjYNuVx7d_PC0VrLo5pcIyf92oLGCluQ9ywjnk_5Xt0IGze4JSNqPTn2Am6mIbarx.V01 Li7Tr1ufc8cOgBIGbDhGIN9nIHLKu8YVn__6jdc8JF96qXznjjlUT49i.9std4wF5YZP7DQL0Fwe KoImymLPININSyWUo.aZ_mIYQkb02Fa19LTHJVlQX4iBbUPvlEvqzbJnRatgFgnIwAKhBaCnzqZq euC59Md.SsnOdpMmmO1Il5.3DSo0gWSBT2drcmwJIlSdKbcl_KwywUe1YIqLmXOVsT0Z1Ox1VRgx y3F9pPN3XFQf2rhsTcUJ0f.T2tiK73JofypPOay7mET3AHFmaCs.vks.G26_itwA99LEuJkE1rJd I4b1Jf0Hbhf4QIuU839hPYUQkfhcZBR3JYqiSWG9fOa5a5LRwvyYxUBlHSeV39lYXXnGeZnWs.8. b6NEfe5mN_F8YBZj6tQQnI8SUa9mc53m.0S2bQiIT7k5ZbWAGY3QYxeCDe7mA9zP1fDIu.26SyOq wf8NZ5ba_g4JaMQRzS2gldJMK3FiwDDNZizFtoihHzOd5fnmTw2dKTXovao4fYKhoQNmnExUxY4V lybGwwL7ROzzBNsZkaFLAgXnTPQTOOqmbP12lKk4_FZdgJHGSBC8utVFtDhKIaDuntr5dcCXmUnB oj4zVISkgLrK360xcfNxJBM8LD8v9RUBCtBpQjNGQR1SN45_TzTVRhXKZ3tNkSvbskJrtvPCfsmN DNKfb_QHOGgREfWqLF9LGZkQ.e3OHSfz0AhGvkB_Tc0XZ.TPAgIQDdUtWHs4JCx48wGvy64gNmHO jgpXMXxxf5RqFRTF1hoD5HpS6Z8XEkgpjQ09WJ4tA1z1ZY1s9No_G2rvJuUbmgYTO2FwV9g9fhsH MyMGwNcGP5Qjn_91DHndR7gc7LFC12DxN7rBCIN8vuYqafFMF04ks9QV9TFRJOgwbf.8SPmVSpxg M.ECpbY88cJdmD2PFp13oQWL1b5mabCBvtMWmyS8bPLStTtqcIDSJNRJHK8U0DCs9GFA.5KpjQfG PCoZucENTIvB9hXD.5JP1vuScNYydoD.PxJp38nbtJjLfWiTlpYiYlQl54dDAGkcafG8xv1esVO3 c5o2tEy7XOWl5muqamKYR3dpTf_3XeTJNgumB8nGIOpDnVkFb6digzxGCbFSjSBvEfsbEU4oZaNI Ol3ZbI7nCtu_fDGRwUMpNku4XNU_eTRhtgMVFrEoshU_fHQzpmSVBZWvAs6OkHdDfjvvtbpFF7.v MoVDl6CbPbNa9ECg4Xi8dmRt0vaYnYiX7lyIez72fNVHbx_HiE83o922FV48QDSc4e8sPIs_D4tB j4rT81_3n6bicbMYJsL_lrLDmQ8R3RmihovYoVH0G_6aGH95yhixd2_QhXC9aWqS2f2o9qeDghth WAfjjVkv2RWJqBSuXyEQGCdiPuZ61UG6lZi55_rjuz.MbGYBHiCfNSMxw Received: from sonic.gate.mail.ne1.yahoo.com by sonic315.consmr.mail.gq1.yahoo.com with HTTP; Wed, 20 Jan 2021 01:42:24 +0000 Received: by smtp405.mail.gq1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 6ab45ba876346170a3cb4d9f66d6453d; Wed, 20 Jan 2021 01:42:23 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.40.0.2.32\)) Subject: Re: Silent hang in buildworld, was Re: Invoking -v for clang during buildworld From: Mark Millard In-Reply-To: Date: Tue, 19 Jan 2021 17:42:22 -0800 Cc: Current FreeBSD , freebsd-arm@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <056845FE-7131-4951-96AF-805D07F7BE0D@yahoo.com> References: <20210116043740.GA19523@www.zefox.net> <20210116155538.GA24259@www.zefox.net> <20210116220334.GA26756@www.zefox.net> <20210117174006.GA30728@www.zefox.net> <85889EAE-F579-4220-9185-944D9AA5075A@yahoo.com> <20210118015009.GA31353@www.zefox.net> <60CCCDE8-E3D3-4920-9FC0-A945330F6830@yahoo.com> <00104FAD-E32B-4DDE-80DD-FCEF14CEC06B@yahoo.com> To: bob prohaska X-Mailer: Apple Mail (2.3654.40.0.2.32) X-Rspamd-Queue-Id: 4DL7Yt2dh6z3H30 X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.43 / 15.00]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-0.93)[-0.932]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RBL_DBL_DONT_QUERY_IPS(0.00)[98.137.65.31:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; SPAMHAUS_ZRD(0.00)[98.137.65.31:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.65.31:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.65.31:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-arm] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Jan 2021 01:42:28 -0000 On 2021-Jan-18, at 21:12, Mark Millard wrote: > On 2021-Jan-18, at 19:19, Mark Millard wrote: >=20 >> . . . >>> FYI: I re-established my access to a RPi2B V1.1 and made >>> it report: "maximum recommended amount (468832 pages)" >>>=20 >>> (The figure can vary some from release to release.) >>>=20 >>> 468832*4096 =3D=3D 1920335872 or a little over 1831 MiBytes >>>=20 >>> For the 4096 Byte pages, that means that the following from >>> gpart fits without complaint (size is in blocks, not pages): >>>=20 >>> 413140992 3686400 da0p2 freebsd-swap (1.8G) >>>=20 >>> 3686400*512 is a little over 1.75 GiByte or 1800 MiByte. So >>> I've left some room below 1831 MiBytes, but not a lot. >>>=20 >>> FYI about my build experiment that is running: >>>=20 >>> # sysctl hw.physmem >>> hw.physmem: 979042304 >>>=20 >>> which, in recent times for armv7, I can (and did) set in >>> /boot/loader.conf on a faster cortex-A7 SBC (that can boot >>> the same media but has more RAM). >>>=20 >>> So I tried a -j4 build, but with LDFLAGS.lld+=3D -Wl,--threads=3D1 >>> in use and my other particular src.conf/make.conf like content >>> (so the builds do likely differ from yours in various ways). >>> My build is producing a non-debug build (but with -g symbols). >>> Somewhat after where your buildworld.log stops, my odd variant >>> of top was reporting: >>>=20 >>> Mem: . . . , 753672Ki MaxObsActive, 200412Ki MaxObsWired, 892732Ki = MaxObs(Act+Wir) >>> Swap: . . . , 145832Ki MaxObsUsed >>>=20 >>> and top was also showing lots of processes as having "0B" RES >>> in STATE "wait" or "nanslp" (so, apparently swapped out, not = paging). >>> ("MaxObs" is short for "maximum observed".) >>>=20 >>> For comparison, your swapscript.log reported a maximum total of >>> 346192 KiBytes "Used" for swap, about 98% into the log file. >>>=20 >>> (Time goes by . . .) >>>=20 >>> It finished with building libllvm and is part way into building >>> libclang. This is probably well past where your hangup happened, >>> given that your published buildworldlog file stopped with >>> libllvm's Target/ARM/ARMMCInstLower.o . My odd top now shows: >>>=20 >>> Mem: . . . , 753672Ki MaxObsActive, 200412Ki MaxObsWired, 892732Ki = MaxObs(Act+Wir) >>> Swap: . . . , 392328Ki MaxObsUsed >>>=20 >>> The build continues to run. I'll let you know how it goes. >>> . . . >>=20 >> Just after libclang finished my odd top showed: >>=20 >> Mem: . . . , 753672Ki MaxObsActive, 200412Ki MaxObsWired, 892736Ki = MaxObs(Act+Wir) >> Swap: . . . , 537588Ki MaxObsUsed >>=20 >> After liblldb: >>=20 >> Mem: . . . , 753672Ki MaxObsActive, 200412Ki MaxObsWired, 899276Ki = MaxObs(Act+Wir) >> Swap: . . . , 537588Ki MaxObsUsed >>=20 >> Much later, after the lldb program had been built: >>=20 >> Mem: . . . , 765700Ki MaxObsActive, 200412Ki MaxObsWired, 954116Ki = MaxObs(Act+Wir) >> Swap: . . . , 537588Ki MaxObsUsed >>=20 >>>>> World build completed on Mon Jan 18 19:10:08 PST 2021 >>>>> World built in 72960 seconds, ncpu: 4, make -j4 >>=20 >> This was building from scratch what was already installed: >>=20 >> # ~/fbsd-based-on-what-freebsd-main.sh=20 >> merge-base: 818390ce0ca539300dd15d7a817784f1e3f7a9b8 >> merge-base: CommitDate: 2021-01-13 21:27:44 +0000 >> 4180404713ec (HEAD -> mm-src) mm-src snapshot for mm's patched build = in git context. >> 818390ce0ca5 (freebsd/main, freebsd/HEAD, pure-src, main) arm64: fix = early devmap assertion >> FreeBSD OPiP2E_RPi2v11 13.0-CURRENT FreeBSD 13.0-CURRENT = mm-src-c255938-g4180404713ec GENERIC-NODBG arm armv7 1300135 1300135 >>=20 >> This suggests that you should be able to build on the RPi2B v1.1, >> using -j4, with appropriate configuration for what and how to build. >>=20 >>=20 >> It is now building the matching kernel, my GENERIC-NODBG style. >=20 > Done: >=20 >>>> Kernel build for GENERIC-NODBG completed on Mon Jan 18 20:33:26 PST = 2021 >>>> Kernel(s) GENERIC-NODBG built in 4998 seconds, ncpu: 4, make -j4 >=20 > So, World+Kernel in somewhat under 22 hours. >=20 > The "MaxObs*" figures were unchanged, so: >=20 > Mem: . . . , 765700Ki MaxObsActive, 200412Ki MaxObsWired, 954116Ki = MaxObs(Act+Wir) > Swap: . . . , 537588Ki MaxObsUsed >=20 > This suggests that, for now, 800 MiByte of swap would be something > more than 1.5 times what it actually used and 1050 MiBytes would > be something like 2.0 times what it actually used, so leaving some > notable margin for variations in peek usage, at least when linker > threading is avoided. >=20 >=20 >=20 > As for what I used to control "what and how to build" . . . >=20 > # more = ~/sys_build_scripts.armv7-host/make_armv7_nodebug_clang_bootstrap-armv7-ho= st.sh=20 > kldload -n filemon && \ > script = ~/sys_typescripts/typescript_make_armv7_nodebug_clang_bootstrap-armv7-host= -$(date +%Y-%m-%d:%H:%M:%S) \ > env __MAKE_CONF=3D"/root/src.configs/make.conf" SRCCONF=3D"/dev/null" = SRC_ENV_CONF=3D"/root/src.configs/src.conf.armv7-clang-bootstrap.armv7-hos= t" \ > WITH_META_MODE=3Dyes \ > WORLD_FLAGS=3D"${WORLD_FLAGS} UBLDR_LOADADDR=3D0x42000000" \ > MAKEOBJDIRPREFIX=3D"/usr/obj/armv7_clang/arm.armv7" \ > make $* >=20 > (In my context, UBLDR_LOADADDR is ignored by anything that > can not use the figure given. So I've no bothered to be > more selective about having it in the armv7 builds.) >=20 > # more ~/src.configs/make.conf > LDFLAGS.lld+=3D -Wl,--threads=3D1 >=20 > # more ~/src.configs/src.conf.armv7-clang-bootstrap.armv7-host > TO_TYPE=3Darmv7 > # > KERNCONF=3DGENERIC-NODBG > TARGET=3Darm > .if ${.MAKE.LEVEL} =3D=3D 0 > TARGET_ARCH=3D${TO_TYPE} > .export TARGET_ARCH > .endif > # > #WITH_CROSS_COMPILER=3D > WITH_SYSTEM_COMPILER=3D > WITH_SYSTEM_LINKER=3D > # > WITH_LIBCPLUSPLUS=3D > WITHOUT_BINUTILS_BOOTSTRAP=3D > WITH_ELFTOOLCHAIN_BOOTSTRAP=3D > #Disables avoiding bootstrap: WITHOUT_LLVM_TARGET_ALL=3D > WITHOUT_LLVM_TARGET_AARCH64=3D > WITH_LLVM_TARGET_ARM=3D > WITHOUT_LLVM_TARGET_MIPS=3D > WITHOUT_LLVM_TARGET_POWERPC=3D > WITHOUT_LLVM_TARGET_RISCV=3D > WITHOUT_LLVM_TARGET_X86=3D > WITH_CLANG=3D > WITH_CLANG_IS_CC=3D > WITH_CLANG_FULL=3D > WITH_CLANG_EXTRAS=3D > WITH_LLD=3D > WITH_LLD_IS_LD=3D > WITHOUT_BINUTILS=3D > # > WITH_LLDB=3D > # > WITH_BOOT=3D > WITHOUT_LIB32=3D > # > # > WITHOUT_WERROR=3D > #WERROR=3D > MALLOC_PRODUCTION=3D > WITH_MALLOC_PRODUCTION=3D > WITHOUT_ASSERT_DEBUG=3D > WITHOUT_LLVM_ASSERTIONS=3D > # > # Avoid stripping but do not control host -g status as well: > DEBUG_FLAGS+=3D > # > WITH_REPRODUCIBLE_BUILD=3D > WITH_DEBUG_FILES=3D > # > # Use of the .clang 's here avoids > # interfering with other CFLAGS > # usage, such as ?=3D usage. > CFLAGS.clang+=3D -mcpu=3Dcortex-a7 > CXXFLAGS.clang+=3D -mcpu=3Dcortex-a7 > CPPFLAGS.clang+=3D -mcpu=3Dcortex-a7 >=20 > (I do not claim that you would want WITH_REPRODUCIBLE_BUILD . > I just happen to have been experimenting with it. You might > not want to be explicit about the cpu to target. You might > not want WITH_CLANG_EXTRAS .) >=20 > # more /usr/fbsd/mm-src/sys/arm/conf/GENERIC-NODBG > include "GENERIC" >=20 > ident GENERIC-NODBG >=20 > makeoptions DEBUG=3D-g # Build kernel with gdb(1) = debug symbols >=20 > options AUDIT # Not enabled by default in = armv7/v6 kernels > # Enabled here to allow kyua = test runs to > # possibly report auditing = works. >=20 > options ALT_BREAK_TO_DEBUGGER >=20 > options KDB # Enable kernel debugger = support >=20 > # For minimum debugger support (stable branch) use: > options KDB_TRACE # Print a stack trace for a = panic > options DDB # Enable the kernel debugger >=20 > # Extra stuff: > #options VERBOSE_SYSINIT=3D0 # Enable verbose sysinit = messages > #options BOOTVERBOSE=3D1 > #options BOOTHOWTO=3DRB_VERBOSE > options ALT_BREAK_TO_DEBUGGER # Enter debugger on keyboard = escape sequence > options KLD_DEBUG > #options KTR > #options KTR_MASK=3DKTR_TRAP > ##options KTR_CPUMASK=3D0xF > #options KTR_VERBOSE >=20 > # Disable any extra checking for. . . > nooptions INVARIANTS # Enable calls of extra sanity = checking > nooptions INVARIANT_SUPPORT # Extra sanity checks of = internal structures, required by INVARIANTS > nooptions WITNESS # Enable checks to detect = deadlocks and cycles > nooptions WITNESS_SKIPSPIN # Don't run witness on = spinlocks for speed > nooptions DEADLKRES # Enable the deadlock resolver > nooptions MALLOC_DEBUG_MAXZONES # Separate malloc(9) zones > nooptions DIAGNOSTIC > nooptions BUF_TRACKING > nooptions FULL_BUF_TRACKING > nooptions USB_DEBUG > nooptions USB_REQ_DEBUG > nooptions USB_VERBOSE >=20 > The /boot/loader.conf file and the /etc/sysctl.conf files > both contained: >=20 > vm.pageout_oom_seq=3D120 > vm.pfault_oom_attempts=3D-1 >=20 > (The hw.physmem=3D979042304 in /boot/loader.conf was very-special, > to better approximate your environment. I also controlled the > cpu frequency used via a line in /etc/sysctl.conf . I do not > bother with such non-default frequency usage [or related settings] > for RPi*'s with the pre-RPi4B style power connections but do > control the frequency for the OPi+2E.) The following had been left implicit about my context and how it manages memory space use. I'll note that I do not use tmpfs or other such memory based file system techniques that could compete for RAM/swap. What is in use for the only file system involved is just the root file system: # df -m Filesystem 1M-blocks Used Avail Capacity Mounted on /dev/gpt/BPIM3root 195378 63940 115808 36% / devfs 0 0 0 100% /dev It is a USB SSD. The swap partition is also on that same media. (The BPIM3 based name dates back to before the BPI-M3 power connection failed and I switched to the OPi+2E.) I'll note that I've started a new from-scratch build without LDFLAGS.lld+=3D -Wl,--threads=3D1 . So at some point I'll have information about how much of a difference (+/-) in swap usage it actually made for with vs. without, if any. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)