From owner-freebsd-arm@freebsd.org Mon Dec 28 20:07:42 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D5A344BFB8B for ; Mon, 28 Dec 2020 20:07:42 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic309-21.consmr.mail.gq1.yahoo.com (sonic309-21.consmr.mail.gq1.yahoo.com [98.137.65.147]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4D4T9n5pZkz4f7Z for ; Mon, 28 Dec 2020 20:07:41 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1609186060; bh=M0pPto+dzpwwupFr9cHTrj+S7tqxElpauFy/BPoKGIG=; h=Subject:From:Date:To:From:Subject; b=oxv5bl0auM2IvsS1B04KMrakiXqSL3vnEfjGV1MZV9NqOEmGik6+LeYCX7EzO+UIQjcM0vJpFeTx+h+6fiAoKwJkjTftcOkqX52S4ds1W7GRgZ+hB+he6706ttn79rVpmG5ffUtTklLcy6QX8HA0fzD3yfFBQjYXlY6MnQSJ2u5QvNGxiseVw/rbkIwKqucMap0HlGTUNbq58/AV6SpdQqGFwjVCzAhnlh4VdnLMtWzXt/dtoZk4kI5IDCfQgM7HfbPn6+6cEPdW2pqWoQibP66vjD5ykqGcGf7aoHrdyx7Qbr0Vrdo2fss6kLr8b9RXObJ2S2sC/gV7ITd2ZqFOKA== X-YMail-OSG: LfW0IaoVM1mG6BF.w96CsSDn4blxjlLhGCUbaLJWvyOCrs9AnIDBswMYBwdnmbf yxS1yWHQoTMoxpwIGx21DBoM8QfOHiXWn86Jw2x9xu_8GFH2_S_b6m3gq5Asm6accG9LWWnzJV68 lutNN5SkcO3dH.nGXXo8ItlJDQeNKrb._0lC43GG53cvboHbhcjPB8l8HWM6ki25Q07vzU0ekwlR E5m1Vxj7VkbuoN2Hydzn4YN3xEoMvAr9.2Y4BrVEb4KkpmzUG3VJM8lw8kVP55cA_ZG2SyLKTk3i TShstu3j_Hbffzp0BKUM.46VOpknO2ghB1P86ecvjBHRZ2wGbg1XpCZXtIihbYluYXva90XXnY9. _lPQACuxajZSeYEB9ASPNP_4dmUOGKRfCmricoVTHgly.MJId66Rrwy5EatbI_SbGQaQNxTPGB79 vVSgRAEwfwPhjX1HjABgJqubVlLdlil8Eb1Ib3hFy_QFBe4mlHePN9A6FwKFkaN_7W5Zr4AOuv3f kLakMIUGl7xRpAzvY1qwaBC1Ue_nVnfp91DSB_Dw3gTXUP5zIEtCPB5REicewvm97fMP7kkEtYxw 4gKW6z5gMbVpMOS9J8BK3BCwnHdC3po7ACeLh5HdBuXkBxDY1xOPXsNDy585hecYjO5.GgyDy01u .Ctbvey2wNPgoPW3SLUtI0JlkB2hNWx7EvEwfXnpYNJCZfoK8aavTYIwA3wIqqiA2y_aBsGp5ipH ztcSENvY12lTm4u9Ax0RUQpMkdbZfyVXT71pP7.dLxo.CQJZKxw9f.VUGWQ.4IYAZjRD6lx6VjIW xDHmBkPldOLdUTCeiyLRcve6I3HfaMrgBPsprbDpNAIBbnErQIl8FESEt_G.aX7NWq54yJyS9zyp 5_wpmkHslmHL72dSecl2hghwB.LShPEwLz.VIKA9MdmPDaNxY3kMJoqdwInlJgaNlkMPOGpd1bv1 LECMQ8Txe50Owhwz50VXCgjklvBDgGYbQtqnSz99n90VLq42q5ieeJLyXnsej4FUPY25s12LkPtV MZWJW9j29C2.o_h16iYoNyDkSn2c2ypeEfsZbqtfae.GFnhx_5SMxoZSm1O9uwaY72TXv4jNC3Hj EgxLlz_scZWnGkn44sJ26e2lhzh73Hrir4MO9.K_KGbGYUR0HDiswSwj2YGHGWYlvZUf7vJSiwoh xuNmO4hY8NMuDcsusrP1gZJCKhp0m9PoCHNJ8qpSRhYSHmY0sTOWwk8TgGAnqf5fsgixgxHWaLDj DiL7gk13Z.bT5lm7PY8d8i2ez_nGhcgf7ZdD4_7s8QuhcsFuKHdfes6dYzJ2SVYhUb64Nmhzy8nt kwJzD2pzLQIgU8ZemRT7f_B7elfP2hJ6akREgeCIdLaQh8vufxWAjJMbZhgR.i0uSrAAZdg4gZ2G JLXfrWcQPQCQ8GyJgw0IgKGICpt3cikstWEEQMt.7tJRaSGiYPpLY_.N57jMkAnJeeZkjQF.9Mzv SqYV3L_ZEdM59HTpp3La91l2lgWQ2uj0qPHvTpcmAXNX9syJFGV218kQIaW1D5OUib_.p.nQA5Wq Ak33PVMebexGnAqbjond5ZDmCxe4vUVXOluR29Ekn6A_uI2djLpfoXF0zugUdf9jmUkkBszGz.uk wOtI7EWCr_awHTXC6v1dvzqFRpJug8uwXDeLiGx9SPUPgzNUh7OBluLFCvEfaDXrI2CxXU1TO4wn c52BOfHHGHgab.rm.wnp_7jN2wA5SaqjimRXV.JWSnzM0ti3Toxex7PXdvlj1eozCMn8abCGQd_j Tgcj3B6ZzoH3whL427yx7lNPRdXypQBabAzJDwrRSDJCVveftwf8HiRUoRYr6kEDnrxCVnrNsEJn YqGrYyKPgLTI8rRaHnIqCWDxVLv6n.Dtih2kuAxS_2gLMeKXYc7s1w9bjCmjhv9_l0uxbtilK096 jnrZQfi_7WsFUkH6Wqraorya26aHMq9kwvXuAfCeDi8cijn84.dFeItehURhsDrLCp_E0V891w45 dPe0HgfKcDePMBAdJ1_m_J_M6.L8iTOuW4D8ZxjfyeXzEelZxNmCeNLQuSSurraypwptM6WOWxnA _.788ARfso8NxUJ29dmpO0zM5em5BDp1Xizy1l091Jiz8Z0sS2FFeRY6OBgOl.KRcXagFyx58yCB fKKj1zD2QGPaejhU8qybUwSnMQ8IimOJt1ZePKr68zog97ZWt9v23BU3FQw5Y00p10iHMY8YhCUV BzJ..NrycZ7.2iqyJaa62hAX0BL9mHCVxjiW1oztg_nZi7E0UW.gGWcySbl_YxtUf4Hoqrxovx9B ojJ5w_IWis8KlG8f0boaWTBKDQjZZhuGfzWh3OH1HU4_VUr4oB1JRKW4rlTpEJtkxf4FjQXzVFTa O9qSNxnZ6LHNhUMnMx9NviKmcNJaAlLK6vUiI6xrv_RebaVPXszKorz4XPHWLzsoslQ5povYlOpd _wJ8Cv9MQgC2sb0qPAyPjyErQIYTLlQdAGFKyR43RYrVOqTor.Q-- Received: from sonic.gate.mail.ne1.yahoo.com by sonic309.consmr.mail.gq1.yahoo.com with HTTP; Mon, 28 Dec 2020 20:07:40 +0000 Received: by smtp401.mail.gq1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 84f51e92301f390bdd3896bd12ae0415; Mon, 28 Dec 2020 20:07:37 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.40.0.2.32\)) Subject: Re: Migrating from -current to stable/12 on RPI2B (ARMv7) From: Mark Millard In-Reply-To: <20201228185622.GB28380@www.zefox.net> Date: Mon, 28 Dec 2020 12:07:36 -0800 Cc: freebsd-arm Content-Transfer-Encoding: quoted-printable Message-Id: References: <20201228044840.GA28380@www.zefox.net> <20201228185622.GB28380@www.zefox.net> To: bob prohaska X-Mailer: Apple Mail (2.3654.40.0.2.32) X-Rspamd-Queue-Id: 4D4T9n5pZkz4f7Z X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.50 / 15.00]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-1.00)[-0.996]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RBL_DBL_DONT_QUERY_IPS(0.00)[98.137.65.147:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; SPAMHAUS_ZRD(0.00)[98.137.65.147:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.65.147:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.65.147:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-arm] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Dec 2020 20:07:42 -0000 On 2020-Dec-28, at 10:56, bob prohaska wrote: > On Sun, Dec 27, 2020 at 10:10:18PM -0800, Mark Millard wrote: >>=20 >>=20 >> On 2020-Dec-27, at 20:48, bob prohaska wrote: >>=20 >>> Having a bit of trouble migrating to stable/12 from 13.0-CURRENT=20 >>> FreeBSD 13.0-CURRENT #3 r368820: Sat Dec 26 19:01:50 PST 2020 =20 >>> bob@www.zefox.com:/usr/obj/usr/src/arm.armv7/sys/GENERIC-MMCCAM arm >>>=20 >>> This is on a Pi2B v1.1 The MMCCAM kernel option was added for sake=20= >>> of the experiment and hasn't, up to now, been accompanied by any >>> visible problems. World and kernel have been built many times, no >>> problems.=20 >>>=20 >>> Buildworld keeps stopping at >>> --- clang.full --- >>> c++ -O -pipe -fno-common -mlong-calls = -I/usr/obj/usr/freebsd-src/arm.armv7/tmp/obj-tools/lib/clang/libclang = -I/usr/obj/usr/freebsd-src/arm.armv7/tmp/obj-tools/lib/clang/libllvm = -I/usr/freebsd-src/contrib/llvm-project/clang/include = -I/usr/freebsd-src/lib/clang/include = -I/usr/freebsd-src/contrib/llvm-project/llvm/include = -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS = -DHAVE_VCS_VERSION_INC -DNDEBUG = -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"armv7-unknown-freebsd12.2-gnueabihf\" = -DLLVM_HOST_TRIPLE=3D\"armv7-unknown-freebsd12.2\" = -DDEFAULT_SYSROOT=3D\"/usr/obj/usr/freebsd-src/arm.armv7/tmp\" = -DLLVM_TARGET_ENABLE_ARM = -DLLVM_NATIVE_ASMPARSER=3DLLVMInitializeARMAsmParser = -DLLVM_NATIVE_ASMPRINTER=3DLLVMInitializeARMAsmPrinter = -DLLVM_NATIVE_DISASSEMBLER=3DLLVMInitializeARMDisassembler = -DLLVM_NATIVE_TARGET=3DLLVMInitializeARMTarget = -DLLVM_NATIVE_TARGETINFO=3DLLVMInitializeARMTargetInfo = -DLLVM_NATIVE_TARGETMC=3DLLVMInitializeARMTargetMC -ffunction-sections = -fdata-sections -gline-tables-only -Wno-for >>> mat-zero-length -Qunused-arguments = -I/usr/obj/usr/freebsd-src/arm.armv7/tmp/legacy/usr/include = -fno-exceptions -fno-rtti -std=3Dc++14 -stdlib=3Dlibc++ = -Wno-c++11-extensions -Wl,--gc-sections -static = -L/usr/obj/usr/freebsd-src/arm.armv7/tmp/legacy/usr/lib -o clang.full = cc1_main.o cc1as_main.o cc1gen_reproducer_main.o driver.o = /usr/obj/usr/freebsd-src/arm.armv7/tmp/obj-tools/lib/clang/libclang/libcla= ng.a = /usr/obj/usr/freebsd-src/arm.armv7/tmp/obj-tools/lib/clang/libllvm/libllvm= .a -L/usr/obj/usr/freebsd-src/arm.armv7/tmp/obj-tools/lib/libz -lz = -L/usr/obj/usr/freebsd-src/arm.armv7/tmp/obj-tools/lib/libexecinfo = -lexecinfo -L/usr/obj/usr/freebsd-src/arm.armv7/tmp/obj-tools/lib/libelf = -lelf = -L/usr/obj/usr/freebsd-src/arm.armv7/tmp/obj-tools/lib/ncurses/ncursesw = -lncursesw -L/usr/obj/usr/freebsd-src/arm.armv7/tmp/obj-tools/lib/libthr = -lpthread -legacy >>> ld: error: failed to open clang.full: Cannot allocate memory >>> c++: error: linker command failed with exit code 1 (use -v to see = invocation) >>> *** [clang.full] Error code 1 >>>=20 >>> make[4]: stopped in /usr/freebsd-src/usr.bin/clang/clang >>> 1 error >>>=20 >>> The failure occurs many minutes after the log entry appears.=20 >>> No errors on the console, swap tops near half a gig. The=20 >>> stable/12 sources being compiled were obtained via git. The=20 >>> -current sources used to compile the running system were=20 >>> obtained using svnlite.=20 >>>=20 >>> I don't recall seeing this much swap used before by armv7,=20 >>> but that's the only notable oddity. =20 >>>=20 >>> The stable/12 sources have been updated every day for the=20 >>> past few in hopes the trouble might go away, the -current=20 >>> sources seem to be updated as far as svnlite goes.=20 >>>=20 >>> Any suggestions appreciated, thanks for reading >>>=20 >>=20 >> I presume the historical system tuning for small memory systems >> from our past investigations and help that we got. >>=20 > Correct. /boot/loader.conf contains=20 > vm.pageout_oom_seq=3D"4096" > vm.pfault_oom_attempts=3D"-1" >=20 >=20 >> You do not mention -j4 vs. -j1 or the like. I'm assuming -j1 has >> been tried. >>=20 > j values of 4, 2 and 1 have been tried, to no effect >=20 >> You have not said if you have adjusted the likes of: >> kern.maxswapzone or VM_SWZONE_SIZE_MAX or if you get the boot >> notices about the likes of: >>=20 >> warning: total configured swap (??? pages) exceeds maximum = recommended amount (??? pages). >>=20 >> Mistuning these could lead to RAM fragmentation issues as I >> understand. >>=20 > Dmesg reports=20 > warning: total configured swap (579552 pages) exceeds maximum recomm > ended amount (467936 pages). > warning: increase kern.maxswzone or reduce amount of swap. > but that hasn't caused recognizable trouble up to now. Adding > even more swap doesn't make things any worse.=20 The question is more if having somewhat less than 467936 pages of swap leads to less problems with allocating possibly sizable contiguous RAM if such is being attempted. (It may not help.) >> Since the allocation failure is reported by ld while it tries to >> produce clang.full , I wonder if you have used: >>=20 >> LDFLAGS.lld+=3D -Wl,--no-threads >>=20 >> in, say, /etc/make.conf ? If not, the llvm ld will try to use >> all 4 RPi2 v1.1 cores (via threads, not via processes) and, from >> what I've seen, will end up trying to use more RAM than when it >> runs single-threaded (main thread only). Of course, links then >> take longer to complete. (LLVM ld actually ends up up with 4+2 >> threads, counting the processes's main thread as well.) >>=20 >> Except on the machines with plenty of RAM (by a large margin for >> the hardware-thread count), I use --no-threads for contexts where >> I can readily control such. >>=20 >> (Ports do not use LDFLAGS.lld and gnu's linker does not tolerate >> --no-threads as a command line option. So "readily" is limited to >> buildkernel and buildworld .) >>=20 > I didn't know about LDFLAGS, but a re-try with=20 > -j1 and LDFLAGS.lld+=3D -Wl,--threads=3D1 > (apparently the syntax changed) in /etc/make.conf=20 > promptly reproduced the error.=20 Hmm. It been a while since I did a native build instead of a cross build. The cross build context has RAM and does not use the assignment so I'd not noticed. Thanks for the report! > At this point the host has two source directories: > /usr/src, obtained using svnlite, holding -current > and owned by root, along with=20 > /usr/freebsd-src, obtained using git, holding -stable/12=20 > owned by bob (member of wheel) where the error is occuring. >=20 > Could this be either a permissions problem or a conflict > among intermediate files left over between make attempts? > Each source directory is matched by its own object directory, > so that seems unlikely. There have been no warnings on the=20 > console related to memory or swap, only the cryptic little=20 > "cannot allocate memory" in the buildworld log. >=20 > It's especially puzzling that the problem is evident only > when compiling stable/12 as a regular user, but not when > compiling -current as root. I've not experimented with regular-user based builds at all, just root based builds with the files own by root. Have you done regular-user based builds before? Is this new behavior for something that used to to work? > I'm trying a sanity check now by deleting clang.full from > /usr/obj/usr/src (two copies) and restarting buildword with > -j4 and no LDFLAGS to verify the error isn't present in > -current. That might take a day or two.=20 =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)