From nobody Mon Jul 15 00:20:55 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WMjXx42wGz5Q2Ks for ; Mon, 15 Jul 2024 00:21:05 +0000 (UTC) (envelope-from philip@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WMjXx1BrSz44tN; Mon, 15 Jul 2024 00:21:05 +0000 (UTC) (envelope-from philip@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1721002865; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=LSPotmti7dJxLawN7Y864gcSlOTRDKwxYVioDVf9DUg=; b=MdKa/EIXuFh4I1S74V1uowTAdABarl/JEx0HeM1ugQYovTA5Nw/Q8OZtEwEhuDvcCtidF7 QJAYKpmBMSVqngAbp3TiFSrYOo1sfxEc/pTi3C/+x9H1B8XAySNED8SLtX2RVE6ILk6JpL wTRTUsmkk7Elkd4L+Y4ymqbsrz2X6XCDoK9Rx0EY/bqnVMd67l1Tl0dxfu8hfcgb2ga2Rh 4z7i8HvysSxfWAmtL46bkK/w+KWiPP7dEREAt9Ai1cr2KOYZ3AJac+7Zrt+/zxRSeTgI9J xge4hb+mYfLrRp1ESJk4yHweQAHWQBD3Ny/FmAuQL3hRCSDMZSK2UN6NkLdooQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1721002865; a=rsa-sha256; cv=none; b=FiaOmVed/dYx5x3WnCibXw5b/JLHYY5DB6VmEWmguNLZNr17ihdYUl6+bW9weeWqD34FMr 6CEZAGus2LJFmbHKqgtwptQJknliT7h5X1gMFxfsb6DiJxY2Yu59M55JcgI7ku5943iHyg CszugGkpiyXrC/0ZoD0lSN5kDAtU/ntC8TisQHey9OQBa+ayUPgI1pnOCuTBlWS1MTxjgM AzWG3l66WLw/EEqGkGHbpaagl8lhvnU13Z0CR7pVVmZE9eQG25T48C/1Px8jvGG5h5MJue hz89qTL/WC5nnypdS3B+/lrj4O5I5TNNQEOOVrO4DtYSSokeIK0nzUa2OM+kXw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1721002865; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=LSPotmti7dJxLawN7Y864gcSlOTRDKwxYVioDVf9DUg=; b=eEj1UxK+M526rI6yGEYX/S4hSB0VQqp0XbQbshUKKQjmWoG68Nv/kZ/IOlmhfR+AGfGPTa VrkwB1wpmoeTQyDEkeaz48Iy/SORZgToKBtDG8WA11mdZZdc0y8E+sH9O7KVO5piTyyUMm 9I4cMpFgOFHsLuA82L8R9wzDryFts3icJhXTYlch8BU5hfuimRUrC25gSOKI+dqYNEIsuH cqlxUDWpDWoMkw9e9HlCCxgnQ67PaBECu6K7Ugs7ZechYlO2kF+X5QIpSTGcJ6ofMc0H7X qnaNgBaTRmRxv9EpAfD1UHOiznyv0tGhWg8wPLhg3sd7raznSaDfXkCBzCRsVw== Received: from fauth1-smtp.messagingengine.com (fauth1-smtp.messagingengine.com [103.168.172.200]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: philip/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4WMjXx0TR2z1297; Mon, 15 Jul 2024 00:21:05 +0000 (UTC) (envelope-from philip@freebsd.org) Received: from compute7.internal (compute7.nyi.internal [10.202.2.48]) by mailfauth.nyi.internal (Postfix) with ESMTP id EBA69120006D; Sun, 14 Jul 2024 20:21:03 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute7.internal (MEProxy); Sun, 14 Jul 2024 20:21:03 -0400 X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeftddrgedugdefhecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvfevufffoffkjghfgggtsehmtdhmreertddtnecuhfhrohhmpefrhhhilhhi phcurfgrvghpshcuoehphhhilhhiphesfhhrvggvsghsugdrohhrgheqnecuggftrfgrth htvghrnhepleehieehteelieeitdegvefhveefleehffdtuefhleelvdfhveehtdefffef vdelnecuffhomhgrihhnpehfrhgvvggsshgurdhorhhgnecuvehluhhsthgvrhfuihiivg eptdenucfrrghrrghmpehmrghilhhfrhhomhepphhhihhlihhpodhmvghsmhhtphgruhht hhhpvghrshhonhgrlhhithihqdduudeiiedviedvgeekqddvfeehudektddtkedqphhhih hlihhppeepfhhrvggvsghsugdrohhrghesthhrohhusghlvgdrihhs X-ME-Proxy: Feedback-ID: ia691475d:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sun, 14 Jul 2024 20:21:01 -0400 (EDT) From: Philip Paeps To: Mark Millard Cc: mmel@freebsd.org, freebsd-arm@freebsd.org Subject: Re: A better alternative to having builds of main-armv7-default fully disabled and last-built be months out of date Date: Mon, 15 Jul 2024 08:20:55 +0800 X-Mailer: MailMate (1.14r6052) Message-ID: <016A1875-EE40-4815-964D-9762597206B7@freebsd.org> In-Reply-To: <274FAB06-4204-415D-941F-69F623E11266@yahoo.com> References: <91287D15-F0A9-4425-A265-B07418B8714D.ref@yahoo.com> <91287D15-F0A9-4425-A265-B07418B8714D@yahoo.com> <816ef6c9-836f-48bb-9f89-b46898431c87@freebsd.org> <274FAB06-4204-415D-941F-69F623E11266@yahoo.com> List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@FreeBSD.org MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=_MailMate_7F94123E-2EFD-4CA3-BB5C-C097F28827B0_=" --=_MailMate_7F94123E-2EFD-4CA3-BB5C-C097F28827B0_= Content-Type: text/plain; format=flowed Sorry for not following up to this thread earlier. I've been occupied elsewhere in the cluster. On 2024-07-07 16:25:32 (+0800), Mark Millard wrote: > On Jul 6, 2024, at 21:35, Michal Meloun > wrote: >> On 07.07.2024 5:42, Mark Millard wrote: >>> main's armv7 packages that are distributed are getting to be months >>> behind because of the build hangups preventing the builds on >>> ampere2. It's worth reinforcing that this only affects main (15-CURRENT). Our stable/13 and stable/14 packages for armv7 are reasonably up to date. Reasonably for a tier-2 architecture anyway. Whatever is causing this, it's only in main. > The only known failures are on ampere2 as far as I know. > As far as I know there is no known way to configure to > match the formal build procedures used on ampere2. According to the current schedule, armv7 builds happen on ampere3, not ampere2: ampere1: - quarterly arm64.aarch64 13.3-RELEASE 133arm64 -a ampere1: - quarterly arm.armv7 releng/13.3 133releng-armv7 -a ampere1: - quarterly arm64.aarch64 14.0-RELEASE 140arm64 -a ampere1: - quarterly arm.armv7 releng/14.0 140releng-armv7 -a ampere2: - default arm64.aarch64 main main-arm64 -a ampere3: - default arm64.aarch64 13.3-RELEASE 133arm64 -a ampere3: - default arm.armv7 releng/13.3 133releng-armv7 -a ampere3: - default arm64.aarch64 14.0-RELEASE 140arm64 -a ampere3: - default arm.armv7 releng/14.0 140releng-armv7 -a I've attached the poudriere.conf from that machine. It's the same one we have on all the builders. >> I've seen some strange live lockups in arm32 jail, but never managed >> to reproduce it. > > On what kind(s) of hardware? > Any kind of relevant context known? In case it helps: ref15-aarch64.freebsd.org (available to all developers) is an identical configuration as ampereX.nyi.freebsd.org. The former has a newer BIOS (for some reason) but that hopefully should not make a difference. If we reach the point where we think the BIOS version matters, I can try to upgrade the BIOS on the ampereXen. smbios.bios.reldate="06/25/2020" smbios.bios.revision="1.14" smbios.bios.vendor="LENOVO" smbios.bios.version="hve104q-1.14" smbios.bios.reldate="05/30/2019" smbios.bios.revision="1.8" smbios.bios.vendor="LENOVO" smbios.bios.version="HVE104J-1.08" Philip --=_MailMate_7F94123E-2EFD-4CA3-BB5C-C097F28827B0_= Content-Disposition: attachment; filename=poudriere.conf Content-Type: text/plain; name=poudriere.conf Content-Transfer-Encoding: quoted-printable # !!!! DO NOT EDIT THIS FILE, IT IS TRACKED IN SVN (https://pm-svn.freebs= d.org/cluster) !!!! # vim: set filetype=3Dsh: ZPOOL=3Dzroot ZROOTFS=3D/poudriere RESOLV_CONF=3D/etc/resolv.conf BASEFS=3D/usr/local/poudriere USE_PORTLINT=3Dno USE_TMPFS=3Dall DISTFILES_CACHE=3D/usr/ports/distfiles SVN_HOST=3Dsvn0.us-east.FreeBSD.org GIT_BASEURL=3Dgit.freebsd.org/src.git GIT_PORTSURL=3Dgit.freebsd.org/ports.git CHECK_CHANGED_OPTIONS=3Dyes CHECK_CHANGED_DEPS=3Dyes NO_RESTRICTED=3Dyes NO_FORCE_PACKAGE=3Dyes URL_BASE=3D"https://pkg-status.freebsd.org/$(hostname -s)" #MAX_MEMORY=3D8 ATOMIC_PACKAGE_REPOSITORY=3Dno PKG_REPO_FROM_HOST=3Dyes SIGNING_COMMAND=3D"ssh pkg-sign" PRIORITY_BOOST=3D"iridium-browser ungoogled-chromium" PRIORITY_BOOST_VALUE=3D20 ALLOW_MAKE_JOBS_PACKAGES=3D"pkg chromium electron29 iridium-browser llvm-= devel qt5-webengine qt6-webengine ungoogled-chromium" TMPFS_BLACKLIST=3D"chromium electron29 iridium-browser ungoogled-chromium= " TMPFS_BLACKLIST_TMPDIR=3D${BASEFS}/data/wrkdirs HTML_JSON_UPDATE_INTERVAL=3D10 USE_PTSORT=3Dno DETERMINE_BUILD_FAILURE_REASON=3Dno # 48 hours for chromium based browsers MAX_EXECUTION_TIME=3D172800 # 2 hours for chromium based browsers = MAX_EXECUTION_TIME_EXTRACT=3D7200 MAX_FILES_RStudio=3D4096 MAX_FILES_RStudio_server=3D4096 MAX_FILES_atom_ide=3D4096 MAX_FILES_traefik=3D2048 MAX_FILES_gatk=3D2048 MAX_FILES_screego=3D8192 MAX_FILES_vscode=3D8192 BUILD_AS_NON_ROOT=3Dno # Use jail src for native-xtools builds XDEV_SRC_JAIL=3Dyes #CCACHE_DIR=3D/ccache calc_builders() { local GB_PER_BUILDER ncpu arcsize physmem swap vmem emem GB_PER_BUILDER=3D12 ncpu=3D$(/sbin/sysctl -n hw.ncpu) PREPARE_PARALLEL_JOBS=3D$((ncpu + 5)) # Tune BUILD_PARALLEL_JOBS based on amount of VM. physmem=3D$(($(/sbin/sysctl -n hw.physmem) / 1024 / 1024 / 1024)) arcsize=3D$(($(/sbin/sysctl -n vfs.zfs.arc_max) / 1024 / 1024 / 1024)) swap=3D$(($(/sbin/sysctl -n vm.swap_total) / 1024 / 1024 / 1024)) vmem=3D$((physmem + swap)) # Calculate "effective" memory; Ignore greedy ARC. emem=3D$((vmem - arcsize)) if [ ${emem} -lt 0 ]; then # The ARC is probably too big. emem=3D$((emem + arcsize)) fi # Give each builder GB_PER_BUILDER BUILD_PARALLEL_JOBS=3D$((emem / GB_PER_BUILDER)) if [ ${BUILD_PARALLEL_JOBS} -gt ${ncpu} ]; then BUILD_PARALLEL_JOBS=3D${ncpu} elif [ ${ncpu} -ge $((2 * BUILD_PARALLEL_JOBS)) ]; then ALLOW_MAKE_JOBS=3Dyes fi if [ $# -gt 0 ]; then echo "calc_builders(): ncpu=3D${ncpu} physmem=3D${physmem} arcsize=3D${= arcsize} JOBS=3D${BUILD_PARALLEL_JOBS}" >&2 fi } calc_builders "$@" #case "$(hostname -s)" in # foul2) # BUILD_PARALLEL_JOBS=3D20 # ;; #esac --=_MailMate_7F94123E-2EFD-4CA3-BB5C-C097F28827B0_=-- From nobody Mon Jul 15 05:02:03 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WMqnT1q92z5QXKW for ; Mon, 15 Jul 2024 05:02:21 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic307-8.consmr.mail.gq1.yahoo.com (sonic307-8.consmr.mail.gq1.yahoo.com [98.137.64.32]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4WMqnS6RxDz4TD5 for ; Mon, 15 Jul 2024 05:02:20 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1721019738; bh=+RYfnj8VaIrbObq7w9ulJ6EJhMx2DZfU1ObfrzP2SkQ=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=t3CoRKvi0S3tp5riOYlJqORKpkGnKJm3mkHJ3LslwCslczCjJCwJQe/0AtiBapnDJ3LmsMLm3KVXYqDVJCXLUoFGb8i6VJS8wnQY4alytcjBARd9RN1L5NqsTihtUfSCmYyKt0KZy9Ok/B3Ol96q6+Gfn6o/j+3llY4RCQ4zFxfXJ+zIvLYKKWexKukSU37PlK+Ds2ewoLPNB1NvF4asB6VkiI918zjqwWAweKCOdNJB+mu5GzTh3+14+1XhUPTyiI6wK4ukPL7L6L3m7cKgel+X8HJfj9AZALX8E2fylBxy4COdgF/27O2DazVxrJnhXlWRJ71mD9LqolJ8U3KXaA== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1721019738; bh=1oWhJN5XxAlygWfFnT0bGZfYjwtGy6wKLB6MlzCmCxp=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=HvW5J/tEQE+0XqhC0JvGw7EXHZfwSDFm6hKXvydhKKvb7ul7o/RWauKJdgb4S5PupWwzrezwJQiOq+Oaw13VPz0o7lFR07kuH46VeMQ2r6jy8eyjdef9GW4DR6kAOtmU/hgAG0PrKV0J7gdra75PJTGexLmcdE7FoPR20AUMHxFRBX3Ie+45ZiRl39CeottapIzvdh6k5gU+yGBL292N5bDNGOPte+xH1XDWUGzNk211LGqOr0kvmVEeWlBx4IeeALLI20qgGam4HBvAEYk79YXdA7DibKfEETtD3g2JUyI/EUz1Me12VUQV5EyE1Ujtu/jhgBx/sPQVrT1NnWXPcQ== X-YMail-OSG: i0vba8EVM1kdphBJjMydF99TwQqvOHUH4ePkCLG3WuTdXh_neEoZkBBN1Tje9P6 JodUbH0kqRmu0rdvq5RBwg_7sFWVXrBdizoJnZ8EpXEmoquz1MM.xNkzFD5UitWHnBDKkzMla9f7 S_AAQZ7nBGslLoLV2s.tOBta9AO33r7jvhXJl2lQKLmwUFQD_P6AJ0Juo09x_m1RJNVkfTor453k grbwRL7kG_FGV2fFJfBAk6kH7rO31lMSa3OP5CMbfMGjM7RstVubDTPzzQzWe0gHjIuD1cf7zIFh yOmf33EWEQ5aUiekEF_mtvZ69IvAGgcuSyZYLrv73XMIi2sPeIOuJAYkAWwRTHLjDJcXfnQjeCA4 VkGNxmQ5S3QuoMerHLRk36w2coyRDZXW_6n4ZFvTOZPJpPsQcTcnYtctun5lBx7VWQGam7muPFLl wBFhirxrdXfv.kP_H9GoNvgaMa50ry2LYHZSAtesSMVcwDwpbwdLQFX6XlRiVcC4A3mH.I1yegxu MtZVvouFz1At.Q8HTJ17fpqGQt10tkbKavwT1vMUP8.TZ0NYOVyJSGKSepaA6JsIBCE1G_jlpqSx vpKYwNR7kR2dtqQr6eC70cjunyMsKX8Pfs.VBCAJz4N.FSQ_52lfsLlypDS0x1n4d63AIGBe.dFw BrbKsRxwer0nCGo.OogiUnHnIK6rooSOINqt_2xbi0iPRFf80MlLA_JTa6K7CqSvNl5KKS9Nf2md X5mKKCZBNJB0FBTfuPIRPUmVAxyKoplVg2.0CpbanniIgNawKkWs_NF5YxjyEC3k07LPlN3yVW.o .AvgfF7bSJc4fyYE5BTxl.bPjB_WgMX.HRauE6dhFAUD0bOr9sF6LpvFOmWNl1NDFBlUtU7j_a19 F.TeRFNULlp79kPN4.XejCuo6CmWVpmZSLZpQi.QqlFAULa8wAHxyjZ0KAb8X6FqqE83Hdu9XLOM sd59TzTkn8hsI12ILHWf_GpW1CHb8Yef7JrkzrdZkROVvMeGMIN10p3qF5ChgI_2hv4VW56hcDnv 38x5fMMuowfGu9qhrBF86T5_n3dqii5SaguLZRfWXrXxHN6a6EIxFi2Qn9x7Nn.zb62PuTinWfDu JRObu_aaMUMtbCO8bSU97K_ce6FGHpXIm_avisTaQSuMLkd5Pyg4DKVmfXs50fwD5RM_myNm.gkE ZR3l7ZNdzc1OubcSa7HDYNbDz6h_kQBZMgGJpK1mCtRamsy1ACQr07OSs.puKBkh0DZ.A0LgMAvi 4PEzcSzUP7Vq52uNzeoF1yDbIN3OHveIBZYi.0Gje1HY5w67ZnX77plwjwJsG57Bz47YfWs6b5ig dQpKl.OfWI0SvIi._k_ymI9ELK1Owll2ohdSgMknp4701kdBfwNPUVLslHYjEqb6yl.ReIy_SPlX jnTYVYsP8.A0fV1.SEZsTV9oD_XENSs81qPsPq6uEhag2ImidWDNOaQleYGgTsTn4yPGg6hsDq3V 250yTiR0QufHTVYH_0Ai1VoOe11htPEIFzBL0uFYoShWoAKzMA6.Wdpmp.mGTaUIeR6ESImYOs05 YqWEh75UTFaMqXimcB9YIzx_OLoXklVqSk.efOGsHrLxWW1NwjsNfXiotqjC2aziwAtv_hziA.fL nrrpIbtmJz2SAO2TQhdu8hzhLYIDnwmgF7CJgL2Abdv5rjSE4oh9VBv2BP1XmwypcpACyfIqTcon JENQXgYbejDmynje5CoDOlJtJmKeqOcWy.E7waWhL8Bjju0sOmZhK1aKfJDukR4d.46EBl3J7JOy d7Qp8Sr33aYAs9wGLofkU_Fu4VPY.hMLP4yp2iMgKDxqISCQZkNZ06.J9sW9scWy2je.vHpoO_gp pG2dXFb_rLbv.cijC94.XBLdEtmMDt6WxjysWPwSjvPWW3pTNXYoSXmC5811TtpHoUbt4vyNQvVw ZLQNuaHGpIVeU6LI_Yd15RVl6hHzMVEeCrxTG8AHk_.F.clOX7ex2nNMgGMzfeVNOxbcCeEajrr5 xdbynaFd1cj4gJMgjE9b6Bu.715sH749q_0mIlNx4mRfci.dVVsjuZbxSCjvB1adTPaWz7m30kEk zJhnh.l0mEL2Z3UaYsVWu1Hol0mE7ZiQ80DPDb9cPVOdRItaf3MOqvLxsd3uNyU556ILXPN68wNT EXFNU5M4J5G_J17bwGQwEODJC_VQe7mPf8Wrl8GgnNhJnOr.dgvbgaPON1xUoWnEQ246khSWNInE SBLqTLiefY2DfMMWH52EjI_wtb0g3OCg0rKeyzbsUp4hoDGex_Djz5hCzWwSvYHKPE_tKqEMv9Gt T9w-- X-Sonic-MF: X-Sonic-ID: 997be25b-48da-4be2-9254-88c3613a3138 Received: from sonic.gate.mail.ne1.yahoo.com by sonic307.consmr.mail.gq1.yahoo.com with HTTP; Mon, 15 Jul 2024 05:02:18 +0000 Received: by hermes--production-gq1-799bb7c8cf-jmg5m (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID fc75b2e08bf39e447b0c5fed227f6257; Mon, 15 Jul 2024 05:02:14 +0000 (UTC) Content-Type: text/plain; charset=utf-8 List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.600.62\)) Subject: Re: A better alternative to having builds of main-armv7-default fully disabled and last-built be months out of date From: Mark Millard In-Reply-To: <016A1875-EE40-4815-964D-9762597206B7@freebsd.org> Date: Sun, 14 Jul 2024 22:02:03 -0700 Cc: FreeBSD ARM List Content-Transfer-Encoding: quoted-printable Message-Id: <6EE65C66-2E1B-4BF4-B166-C3475E15F3C9@yahoo.com> References: <91287D15-F0A9-4425-A265-B07418B8714D.ref@yahoo.com> <91287D15-F0A9-4425-A265-B07418B8714D@yahoo.com> <816ef6c9-836f-48bb-9f89-b46898431c87@freebsd.org> <274FAB06-4204-415D-941F-69F623E11266@yahoo.com> <016A1875-EE40-4815-964D-9762597206B7@freebsd.org> To: Philip Paeps X-Mailer: Apple Mail (2.3774.600.62) X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] X-Rspamd-Queue-Id: 4WMqnS6RxDz4TD5 On Jul 14, 2024, at 17:20, Philip Paeps wrote: > Sorry for not following up to this thread earlier. > I've been occupied elsewhere in the cluster. >=20 > On 2024-07-07 16:25:32 (+0800), Mark Millard wrote: >> On Jul 6, 2024, at 21:35, Michal Meloun = wrote: >>> On 07.07.2024 5:42, Mark Millard wrote: >>>> main's armv7 packages that are distributed are getting to be months >>>> behind because of the build hangups preventing the builds on = ampere2. >=20 > It's worth reinforcing that this only affects main (15-CURRENT). Our = stable/13 and stable/14 packages for armv7 are reasonably up to date. = Reasonably for a tier-2 architecture anyway. Whatever is causing this, = it's only in main. >=20 >> The only known failures are on ampere2 as far as I know. >> As far as I know there is no known way to configure to >> match the formal build procedures used on ampere2. >=20 > According to the current schedule, armv7 builds happen on ampere3, not = ampere2: >=20 > ampere1: - quarterly arm64.aarch64 13.3-RELEASE 133arm64 -a > ampere1: - quarterly arm.armv7 releng/13.3 133releng-armv7 -a > ampere1: - quarterly arm64.aarch64 14.0-RELEASE 140arm64 -a > ampere1: - quarterly arm.armv7 releng/14.0 140releng-armv7 -a > ampere2: - default arm64.aarch64 main main-arm64 -a > ampere3: - default arm64.aarch64 13.3-RELEASE 133arm64 -a > ampere3: - default arm.armv7 releng/13.3 133releng-armv7 -a > ampere3: - default arm64.aarch64 14.0-RELEASE 140arm64 -a > ampere3: - default arm.armv7 releng/14.0 140releng-armv7 -a Putting the ones that mention armv7 together, with the others omited: ampere1: - quarterly arm.armv7 releng/13.3 133releng-armv7 -a ampere1: - quarterly arm.armv7 releng/14.0 140releng-armv7 -a ampere3: - default arm.armv7 releng/13.3 133releng-armv7 -a ampere3: - default arm.armv7 releng/14.0 140releng-armv7 -a None of those are for main [so: 15]. They are all for working contexts. main-armv7-default last ran on ampere2 2024-05-31/2024-06-01 or so. I'm not aware of any main-armv7-default builds done via ampere1 or ampere3. > I've attached the poudriere.conf from that machine. It's the same one = we have on all the builders. Is the poudriere.conf content the same as for the main [so: 15] context (ampere2) and the ampere3 context(s)? >>> I've seen some strange live lockups in arm32 jail, but never managed = to reproduce it. >>=20 >> On what kind(s) of hardware? >> Any kind of relevant context known? >=20 > In case it helps: ref15-aarch64.freebsd.org (available to all = developers) is an identical configuration as ampereX.nyi.freebsd.org. = The former has a newer BIOS (for some reason) but that hopefully should = not make a difference. If we reach the point where we think the BIOS = version matters, I can try to upgrade the BIOS on the ampereXen. >=20 > smbios.bios.reldate=3D"06/25/2020" > smbios.bios.revision=3D"1.14" > smbios.bios.vendor=3D"LENOVO" > smbios.bios.version=3D"hve104q-1.14" >=20 > smbios.bios.reldate=3D"05/30/2019" > smbios.bios.revision=3D"1.8" > smbios.bios.vendor=3D"LENOVO" > smbios.bios.version=3D"HVE104J-1.08" Looking at the poudriere.conf example, it points out another difference for my more recent testing: strictly UFS contexts for my aarch64 and armv7 systems these days. The only media that is ZFS based these days for any system in my active use is for the 7950X3D (amd64). My switching to UFS matches up with my switching to use pkgbase to install and test official FreeBSD builds (all of: kernel, world, ports) for comparison/contrast with my personal builds of such (that involves some locally patched files). We do know when the last successful from-scratch "bulk -a" involving graphics/graphviz was before the armv7 problems started: the build of pkg started on Feb 19: pe9c9c73181b5_sbd45bbe440 =3D>> Building ports-mgmt/pkg build started at Mon Feb 19 12:47:46 UTC 2024 port directory: /usr/ports/ports-mgmt/pkg package name: pkg-1.20.9_1 building for: FreeBSD main-armv7-default-job-01 15.0-CURRENT FreeBSD = 15.0-CURRENT 1500014 arm maintained by: pkg@FreeBSD.org Makefile datestamp: -rw-r--r-- 1 root wheel 2311 Feb 1 01:02 = /usr/ports/ports-mgmt/pkg/Makefile Ports top last git commit: e9c9c73181b Ports top unclean checkout: no Port dir last git commit: f7f4c1a0472 Port dir unclean checkout: no Poudriere version: poudriere-git-3.4.1 Host OSVERSION: 1500006 Jail OSVERSION: 1500014 Job Id: 01 We also know the first observed failure with the symptoms (not a from-scratch build), where it started with a dns/public_suffix_list build that was dated Feb 28: p43e3af5f5763_sf5f08e41aa =3D>> Building dns/public_suffix_list build started at Wed Feb 28 16:05:30 UTC 2024 port directory: /usr/ports/dns/public_suffix_list package name: public_suffix_list-20240130 building for: FreeBSD main-armv7-default-job-07 15.0-CURRENT FreeBSD = 15.0-CURRENT 1500014 arm maintained by: sunpoet@FreeBSD.org Makefile datestamp: -rw-r--r-- 1 root wheel 770 Feb 25 01:02 = /usr/ports/dns/public_suffix_list/Makefile Ports top last git commit: 43e3af5f576 Ports top unclean checkout: no Port dir last git commit: 906be52cfb7 Port dir unclean checkout: no Poudriere version: poudriere-git-3.4.1-1-g1e9f97d6 Host OSVERSION: 1500006 Jail OSVERSION: 1500014 Job Id: 07 Bisection of the kernel/world combinations between would be very disruptive to other uses of the machine doing the bisections. But such would be one way of trying to narrow down what change(s) lead to the problem showing up for main [so: 15]. So for FreeBSD kernel/world main that would be over: =E2=80=A2 git: bd45bbe440f1 - main - rescue: Fix after zfsbootcfg = addition Warner Losh=20 Tue, 13 Feb 2024 . . . Sun, 25 Feb 2024 . . . =E2=80=A2 git: f5f08e41aa57 - main - loader/efi: Only include = interpreter's linker script Warner Losh Looks like that is something like around 120 commits to main [so: 15]. But for _sbd45bbe440 and _sf5f08e41aa I'm not so sure that the kernel booted matches the system commits referenced. If not, the specific kernel build does not seem to be identified in anything that I have access to. Nothing like the output from the likes of: # uname -v FreeBSD 15.0-CURRENT main-n270963-609cdb12b962 GENERIC is in the build log output (presumes a context with UNAME_v not overriding what would be shown for the specific output). For main, freebsd-version output is not appropriately detailed of an identification for the purpose. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Tue Jul 16 17:42:12 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WNmc31nXCz5Rg6R for ; Tue, 16 Jul 2024 17:42:27 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic317-20.consmr.mail.gq1.yahoo.com (sonic317-20.consmr.mail.gq1.yahoo.com [98.137.66.146]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4WNmc20LQ2z4jMp for ; Tue, 16 Jul 2024 17:42:25 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=jUDMGU1F; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.66.146 as permitted sender) smtp.mailfrom=marklmi@yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1721151743; bh=Hmxp8p0vKrI072VbtnLMqT9fxlgChtKtBTBJ+HATZWY=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=jUDMGU1F+8gBSWlyMGLcdqzTePLQYQZZHOBkQ6RDyMH2dFEdN7Am/YknxyFbmWOqabEafypMU0bg2dpSgFVsecuFc7SCaTT44M01/UnijNQ2r5Ll4ZS6vPUNYHHxp9Ra2phSBxU/i9387BpJ+csRX2xNY1mCUBi/0IIbdWWLws36XzdVHWpKc6z7lI+8KbS23NbbycSgPSbq6PKRFtcGVFZy0bR6xdD/RK5aLS9j/D87GRVeKCKzQz78DEMq8qT/S5S05q1sCuyg+Ri/wOHo9eW3kcmteQZN6fhkXWbZ+/kV/AjcQcwIFLtKTsZGQm37Ui/r/+yZ6yi27fWxrKpp/A== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1721151743; bh=TEUIIRJVaQMhEi8uju4srntKxCUedIKkHcMMWAKddnr=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=aa5VNjf9J/Sp5H3q70wUuD30s18ylbuctbpxuUkaxtrsLGDHXRCD7MmxKb6eGWvw4Y4xL4f1j13s4ifnLoojIUYJcZYTM2E3Tqew0huFlzJCP+hMk8N3M2Cf4GCDD0s/HRcsYSeYENMdctmKmsegR5LH6TgEyY5TBkZk3ZEowYxhwBS90nL2F0DAR3W4vaZt7xCnSiMfDLOM+W20bUdVFkjJ0f6FvJZGBt0n4ZExou+r76+mjB/l4ZxPPcjOUDtKRLgr5UYoT5zDqyA9rAbe6aBu8umlYD2cmioPH1Da3ybzc7PSzW1kBcWLanfe1R3Ps2fU2WM9vsbHLJvXI5j/uw== X-YMail-OSG: syGnqL8VM1nk7JfTGOaE6eJSmy2LHYifMok7aR48aNrPWzkKqQQYEKPlNP86rHs xhxRtsn.IDzjU7USLAjh6b1pT5o1hPm3GZsJyIpDNzznfl2KkgxoQwgWWFxg4zVjheEnveA5V2GJ LY.bs8Tv7urMz92ZxSKCdJgjPpJrcCLSgd06.COaHrQV9SbPGJoanu80mD11E0WwC0R5WGtCoZaZ X03HXjZ6RWak03uRkfssLKcOu9Y0XrBwBCP6ijnS5qbSgsq.HO9xodEWPQCAhnGXB50rGbGWK1ov paCqWmRWDv7xIyuVFXswnC1Z3GHN9WxgGEZLEhM5zl0rKnrXXOkgsszOvf1TRmLH1mF7kHhqy1Mk SXyYn5YW4oswiCcrgPDUpySUhoB7V6yPgCHQEWLwQvkrwdLqotd7GFzABFoQWOJUaVuj3UzenEhY fupx256Ja0sEK9F9L1wT8OxRxvWy49NleuYs6CoPCoTaunSz4KJVymx3Kz4my8S3ecDFWbacDYMv UCvP.BkuuwA6Ki8XVJp4E3VwIouKfQ0cZHayEv4RmwHZcy6_e55wHkPWDSYHP_.mEnpvr5axVCHE 2AuDkRI197gsM5quwADQlZqilKRqRBuT2U8SsErAaVUVYNN8vZdLyfco23jlAsg4lbiuVwHp0PYS BwcVoP4K22xvhO6BuMWjrPhS3TAcRwtYJoPM9vCcNs8Q4_8c6WnC_kXAz7h6cLqn5ZubP6nb6HXY Zsf7eySDlXcapYKJGaPpDgWU._eEwC2LGInTTdQw1FkfOAzi_VYOV5ZZ0c5FeeVkr_D6E7g7aD2K W9R4BURTkWkuIHjwl21bmnAN_fEWOtRPbZYQDa.w1gq9HZ1nfz8JYW.kfd4GmsCZuq0Ry5n3JdMZ HeTU_gYjhf5EKn.QdfttAx9u0ubeb8pO.XBzCgeYBnk5.Qc1l72Q.y4TzHhbvdMKx8FLflCqQ51o pY.O8HAsaOMSXBmbobc7f6DswLISg8tgHxAafbRNHA9x6oYZIKiN.D4HhC5GMxBkPALZ2MsfEOtj lOhjakwmumOdZSXru.NPVrLqJGn6I0Hxx7oOBr18eYRfIfGsIL6ekWTG9e7L3mioSDw8VL1Zzj3E OgjHIZjp8LYobyvfvJTn42Z4uTROf5VPT07WRljgZ9xZe2kGt2ppRRv0WQXglD3vzNHsq6m2YeCn MW0Mo0oLo1kGkF9KG7rXE67rx0aM9zD1S4OmGoa19xb6lTxe7fPtB1pwz8Q4ScSYepOz.tU1Mkce rqrzgwdGmqcaAgoUFKyoIsRr2i.BpSWpPmuo7Z7cFhrvxVe_CJgWowCV7bcxr4FUmJrF9Iahvas4 eRrVHljrr7KXR5yAKwLYNz3GN45TaXcBxXwWUQi0YRNNB8emtblvme1hIj4PsKOlhlyE97.CxgN8 dKw6V5AB3jW8PEJ2lCmAA3BOLE1CwrSSjupjWbUxCe.Q5CZ5TNaU.zCDowKheUYpO9gRQrxL0fjJ U60hVnUbz2pn15xTkevfJgg6d8WLvZ8iG4YLAPJAAuSEBUI81zgjX2vCkXFE2g.5a8bmrHA4phdy xMtvfk0Mz0cRvPSDs1eXiHwBTG.vIgcEuCWD4.L4k92Hbh1.y7MSpyQ5Kx5gYIr4OqaiCNT4K.21 vT3WT_Qy4UzoSZJ81.pfg_GiJkG8.kYCDMng5oMacRfkNekw8Z.43p4r8GP.h37D59iCFE5Epujf APPqCx6mu2i1F0UbJf7SmoliKii6tlJ5Y.nhbYym_3hsdL9x0HspVYOUAHJxvsQCMMf3Aak2_MEJ lUjF6ZFTrFuQoQ0HNb1WYynwg3Ow5ROXkic.GLA0rG3z3vlllrRWJtpLl5FY8.rTv3zQsyVTcyhx .DbdRJYTYo3k3DbSwYmXyFBs.r2J7DDqlXu7TA_9ZY.AliPaZ9xo5fVg.mCmb9xkmGr.WkxB0fd0 i3OqBiK_8t.nMpNoxcm8ZlMmOD8kgOXme7xQdLtJqmGoUVBl4EVoYuEvs1l6_hQMLqWA59P2s2aH qQ0yNNlNogi0.nkbLQJLdeE4uGSuBC6oGAyKng6TM8DWhDR5zwZ8wY4LMxUa8rC18WLi.LUwSCWZ jWwW.NAQMaq5iIye9tLCmX09CZDvGRcIlALMOp1evHAwqU3Ws8wubpcrLvub6aama1LL01GirzE5 gnV3DVh_5rEDYcs7f5kMR0g0H0qszqX7YM.gPKEZCvumxaXEnqfoJ_Uphp0.GyaTpqT8v0iyNh35 C1hWke9dCqQtj.1qEtro9KnavkUnrKsvj.vQXrqmPPRpXJtIr.40jQ7rrx4DJUWhBo2pkqEbJqlH p52A- X-Sonic-MF: X-Sonic-ID: 4bd5ef99-464a-48f2-a431-8902fe66893d Received: from sonic.gate.mail.ne1.yahoo.com by sonic317.consmr.mail.gq1.yahoo.com with HTTP; Tue, 16 Jul 2024 17:42:23 +0000 Received: by hermes--production-gq1-799bb7c8cf-mlhxz (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 7b5c8338a31d5f798b36ae2f31cdeda6; Tue, 16 Jul 2024 17:42:22 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.600.62\)) Subject: armv7-on-aarch64 stuck at urdlck: I got a replication of the bulk build hangup problem on a Windows DevKit 2023 Message-Id: Date: Tue, 16 Jul 2024 10:42:12 -0700 To: Philip Paeps , FreeBSD ARM List , FreeBSD Mailing List , Current FreeBSD X-Mailer: Apple Mail (2.3774.600.62) References: X-Spamd-Bar: --- X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; DKIM_TRACE(0.00)[yahoo.com:+]; TO_DN_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; RCPT_COUNT_THREE(0.00)[4]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MLMMJ_DEST(0.00)[freebsd-arm@freebsd.org]; APPLE_MAILER_COMMON(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.66.146:from]; RCVD_IN_DNSWL_NONE(0.00)[98.137.66.146:from] X-Rspamd-Queue-Id: 4WNmc20LQ2z4jMp No longer is the problem only observed on ampere2! But this was with a non-debug, personally built kernel that has some of my now patches. I'll see if I can replicate the issue with an official pkgbase debug kernel. FYI for the replication that I got: /usr/local/sbin/pkg-static add -A /packages/All/graphviz-9.0.0_4.pkg runs: /usr/local/bin/dot -c each such /usr/local/bin/dot is stuck at MWCHAN urdlck . # poudriere status -b [main-armv7-poud-default] [2024-07-16_04h27m31s] [parallel_build] = Queued: 449 Built: 433 Failed: 0 Skipped: 0 Ignored: 0 Fetched: 0 = Tobuild: 16 Time: 04:44:38 ID TOTAL ORIGIN PKGNAME = PHASE PHASE TMPFS CPU% MEM% [01] 00:59:45 graphics/rubygem-ruby-graphviz | = rubygem-ruby-graphviz-1.2.5 run-depends 00:59:30 1.59 GiB 0% 0.2% [02] 00:49:59 graphics/p5-GraphViz | p5-GraphViz-2.25 = build-depends 00:49:53 1.45 GiB 0% 0.2% [03] 00:59:45 graphics/py-pydot@py311 | py311-pydot-2.0.0 = run-depends 00:59:34 1.47 GiB 0% 0.2% [04] 00:59:45 graphics/py-pygraphviz@py311 | py311-pygraphviz-1.6 = lib-depends 00:59:33 1.47 GiB 0% 0.2% [05] 00:58:57 graphics/py-graphviz@py311 | py311-graphviz-0.10.1 = run-depends 00:58:49 1.47 GiB 0% 0.2% [06] 00:59:20 audio/ganv | ganv-1.8.2_1 = lib-depends 00:59:10 1.53 GiB 0% 0.2% [07] 00:59:45 devel/libr3 | libr3-1.0.0_2 = lib-depends 00:59:19 1.54 GiB 0% 0.3% [08] 00:59:45 net/netmap | netmap-0.1.3_2 = run-depends 00:59:22 1.46 GiB 0% 0.3% I had started the bulk build via the list: # more ~/origins/ampere2-failures-armv7.txt audio/ganv devel/doxygen devel/libr3 graphics/p5-GraphViz graphics/p5-GraphViz2 graphics/oyranos graphics/pear-Image_GraphViz@php81 graphics/py-graphviz@py311 graphics/py-pydot@py311 graphics/py-pygraphviz@py311 graphics/rubygem-ruby-graphviz math/ggobi net-mgmt/librenms net/netmap print/dot2tex@py311 # poudriere bulk -j main-armv7-poud `cat = ~/origins/ampere2-failures-armv7.txt` . . . [00:00:12] Building 449 packages using up to 8 builders . . . [03:44:55] [01] [00:18:54] Finished graphics/graphviz | = graphviz-9.0.0_4: Success [03:44:56] [01] [00:00:00] Building graphics/rubygem-ruby-graphviz | = rubygem-ruby-graphviz-1.2.5 [03:44:56] [03] [00:00:00] Building graphics/py-pydot@py311 | = py311-pydot-2.0.0 [03:44:56] [04] [00:00:00] Building graphics/py-pygraphviz@py311 | = py311-pygraphviz-1.6 [03:44:56] [07] [00:00:00] Building devel/libr3 | libr3-1.0.0_2 [03:44:56] [08] [00:00:00] Building net/netmap | netmap-0.1.3_2 [03:45:21] [06] [00:09:33] Finished x11-toolkits/gtkmm24 | = gtkmm24-2.24.5_4: Success [03:45:21] [06] [00:00:00] Building audio/ganv | ganv-1.8.2_1 [03:45:44] [05] [00:13:33] Finished graphics/ImageMagick6@nox11 | = ImageMagick6-nox11-6.9.12.77_9,1: Success [03:45:44] [05] [00:00:00] Building graphics/py-graphviz@py311 | = py311-graphviz-0.10.1 [03:54:42] [02] [00:24:53] Finished print/texlive-base | = texlive-base-20240312: Success [03:54:42] [02] [00:00:00] Building graphics/p5-GraphViz | = p5-GraphViz-2.25 In /usr/src/sys/kern/kern_umtx.c there is: static int do_rw_rdlock(struct thread *td, struct urwlock *rwlock, long fflag, struct _umtx_time *timeout) { . . . /* * Contention bit is set, before sleeping, increase * read waiter count. */ rv =3D fueword32(&rwlock->rw_blocked_readers, &blocked_readers); if (rv =3D=3D 0) rv =3D suword32(&rwlock->rw_blocked_readers, blocked_readers + 1); if (rv =3D=3D -1) { umtxq_unbusy_unlocked(&uq->uq_key); error =3D EFAULT; break; } while (state & wrflags) { umtxq_lock(&uq->uq_key); umtxq_insert(uq); umtxq_unbusy(&uq->uq_key); error =3D umtxq_sleep(uq, "urdlck", timeout =3D=3D= NULL ? NULL : &timo); umtxq_busy(&uq->uq_key); umtxq_remove(uq); umtxq_unlock(&uq->uq_key); if (error) break; rv =3D fueword32(&rwlock->rw_state, &state); if (rv =3D=3D -1) { error =3D EFAULT; break; } } . . . For reference: # ps -alxdww | less UID PID PPID C PRI NI VSZ RSS MWCHAN STAT TT TIME = COMMAND . . . 0 87700 4522 6 20 0 16576 1888 - T 0 0:00.01 | | = |-- vi /usr/local/share/poudriere/jail.sh 0 91496 4522 4 20 0 15760 4684 select S+ 0 0:06.88 | | = `-- /usr/local/libexec/poudriere/sh -e -o pipefail = /usr/local/share/poudriere/bulk.sh audio/ganv devel/doxygen devel/libr3 = graphics/p5-GraphViz graphics/p5-GraphViz2 graphics/oyranos = graphics/pear-Image_GraphViz@php81 graphics/py-graphviz@py311 = graphics/py-pydot@py311 graphics/py-pygraphviz@py311 = graphics/rubygem-ruby-graphviz math/ggobi net-mgmt/librenms net/netmap = print/dot2tex@py311 0 37688 91496 0 68 0 15760 4700 wait I 0 0:00.05 | | = |-- sh: poudriere[main-armv7-poud-default][01]: build_pkg = (rubygem-ruby-graphviz-1.2.5) (sh) 0 47568 37688 0 68 0 6664 3664 wait IJ 0 0:00.03 | | = | `-- /usr/bin/make -C /usr/ports/graphics/rubygem-ruby-graphviz = run-depends 0 47598 47568 6 68 0 5568 2988 wait IJ 0 0:00.01 | | = | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh 0 47743 47598 6 68 0 11188 8864 wait IJ 0 0:00.00 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg 0 47747 47743 3 20 0 71692 48984 select SJ 0 0:04.26 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg 0 56383 47747 1 68 0 26292 23812 urdlck IJ 0 0:00.02 | | = | `-- /usr/local/bin/dot -c 0 37700 91496 6 68 0 15760 4700 wait I 0 0:00.04 | | = |-- sh: poudriere[main-armv7-poud-default][03]: build_pkg = (py311-pydot-2.0.0) (sh) 0 45102 37700 2 68 0 6668 3704 wait IJ 0 0:00.02 | | = | `-- /usr/bin/make -C /usr/ports/graphics/py-pydot FLAVOR=3Dpy311 = run-depends 0 45156 45102 4 68 0 5584 2992 wait IJ 0 0:00.01 | | = | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh 0 45215 45156 4 68 0 11144 8864 wait IJ 0 0:00.00 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg 0 45218 45215 4 20 0 51420 31512 select SJ 0 0:02.68 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg 0 52147 45218 2 68 0 26292 23812 urdlck IJ 0 0:00.02 | | = | `-- /usr/local/bin/dot -c 0 37721 91496 1 68 0 15760 4700 wait I 0 0:00.04 | | = |-- sh: poudriere[main-armv7-poud-default][04]: build_pkg = (py311-pygraphviz-1.6) (sh) 0 45937 37721 1 68 0 6684 3744 wait IJ 0 0:00.03 | | = | `-- /usr/bin/make -C /usr/ports/graphics/py-pygraphviz FLAVOR=3Dpy311 = lib-depends 0 46009 45937 7 68 0 5584 2992 wait IJ 0 0:00.01 | | = | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh 0 46127 46009 7 68 0 11144 8864 wait IJ 0 0:00.00 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg 0 46129 46127 1 20 0 51384 31548 select SJ 0 0:02.73 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg 0 53311 46129 4 68 0 26292 23812 urdlck IJ 0 0:00.02 | | = | `-- /usr/local/bin/dot -c 0 37744 91496 7 45 0 15760 4692 wait I 0 0:00.04 | | = |-- sh: poudriere[main-armv7-poud-default][07]: build_pkg = (libr3-1.0.0_2) (sh) 0 55198 37744 0 50 0 6664 3664 wait IJ 0 0:00.03 | | = | `-- /usr/bin/make -C /usr/ports/devel/libr3 lib-depends 0 55229 55198 0 68 0 5588 2988 wait IJ 0 0:00.01 | | = | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh 0 55594 55229 7 68 0 11168 8864 wait IJ 0 0:00.00 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg 0 55596 55594 2 20 0 69796 50180 select SJ 0 0:04.53 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg 0 62753 55596 0 68 0 26292 23812 urdlck IJ 0 0:00.02 | | = | `-- /usr/local/bin/dot -c 0 37763 91496 4 29 0 15760 4696 wait I 0 0:00.05 | | = |-- sh: poudriere[main-armv7-poud-default][08]: build_pkg = (netmap-0.1.3_2) (sh) 0 51054 37763 6 36 0 6636 3684 wait IJ 0 0:00.03 | | = | `-- /usr/bin/make -C /usr/ports/net/netmap run-depends 0 51107 51054 3 68 0 5568 2988 wait IJ 0 0:00.01 | | = | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh 0 51576 51107 3 68 0 11168 8860 wait IJ 0 0:00.00 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg 0 51580 51576 3 20 0 68220 49432 select SJ 0 0:04.27 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg 0 59063 51580 6 68 0 26292 23808 urdlck IJ 0 0:00.02 | | = | `-- /usr/local/bin/dot -c 0 53709 91496 1 68 0 15760 4700 wait I 0 0:00.04 | | = |-- sh: poudriere[main-armv7-poud-default][06]: build_pkg = (ganv-1.8.2_1) (sh) 0 63371 53709 6 68 0 6636 3668 wait IJ 0 0:00.03 | | = | `-- /usr/bin/make -C /usr/ports/audio/ganv lib-depends 0 63377 63371 1 68 0 5580 2996 wait IJ 0 0:00.01 | | = | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh 0 63413 63377 4 68 0 11180 8864 wait IJ 0 0:00.00 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg 0 63414 63413 3 20 0 56212 35980 select SJ 0 0:02.21 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg 0 65577 63414 6 68 0 26292 23812 urdlck IJ 0 0:00.01 | | = | `-- /usr/local/bin/dot -c 0 63365 91496 3 68 0 15760 4696 wait I 0 0:00.03 | | = |-- sh: poudriere[main-armv7-poud-default][02]: build_pkg = (p5-GraphViz-2.25) (sh) 0 63807 63365 7 68 0 6696 3672 wait IJ 0 0:00.02 | | = | `-- /usr/bin/make -C /usr/ports/graphics/p5-GraphViz build-depends 0 63808 63807 2 68 0 5568 2988 wait IJ 0 0:00.01 | | = | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh 0 63833 63808 4 68 0 11188 8864 wait IJ 0 0:00.00 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg 0 63834 63833 7 20 0 67400 48532 select SJ 0 0:03.52 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg 0 66041 63834 2 68 0 26292 23812 urdlck IJ 0 0:00.02 | | = | `-- /usr/local/bin/dot -c 0 69974 91496 1 68 0 15760 4700 wait I 0 0:00.04 | | = |-- sh: poudriere[main-armv7-poud-default][05]: build_pkg = (py311-graphviz-0.10.1) (sh) 0 73474 69974 5 68 0 6684 3740 wait IJ 0 0:00.02 | | = | `-- /usr/bin/make -C /usr/ports/graphics/py-graphviz FLAVOR=3Dpy311 = run-depends 0 73496 73474 6 68 0 5584 2992 wait IJ 0 0:00.01 | | = | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh 0 73521 73496 7 68 0 11144 8864 wait IJ 0 0:00.00 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg 0 73522 73521 0 20 0 52432 32664 select SJ 0 0:02.70 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg 0 76540 73522 3 68 0 26292 23812 urdlck IJ 0 0:00.02 | | = | `-- /usr/local/bin/dot -c 0 91907 91496 5 68 0 15760 4492 nanslp S 0 1:05.17 | | = |-- sh: poudriere[main-armv7-poud-default]: html_json_main (sh) 0 99134 91496 1 40 0 15760 4740 piperd I 0 0:03.22 | | = `-- sh: poudriere[main-armv7-poud-default]: pkg_cacher_main (sh) 0 23 =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Tue Jul 16 18:37:34 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WNnqz0RvXz5Rl5c for ; Tue, 16 Jul 2024 18:37:51 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic317-20.consmr.mail.gq1.yahoo.com (sonic317-20.consmr.mail.gq1.yahoo.com [98.137.66.146]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4WNnqy1C4wz4r3R for ; Tue, 16 Jul 2024 18:37:49 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=N+4Rieak; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.66.146 as permitted sender) smtp.mailfrom=marklmi@yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1721155066; bh=4CYAqwtG4OMuw2FxJonNhI/FJYWNrpDrxU9mGWvcmcI=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=N+4RieakXMumE1oiS70cmPF1Mbf56qYHs4eF0oWs7Ce5JhKn8B3N6MU9IJq9OmVEr8cdLKESmGvMHp0CFATXLOvheFR4NSg6FiQU2/cTg9t3x4AQ0RnQ5+bjkcor6JzzN2RsvqU2K9MIFW7mZXaK8FNYwArt4ADAHkeiGfFv6+5eAXdTcVsb962xHdVlg8KxJxB7Qfn1kShvhQyqt5DvN9hSgCKD+oGlsv9GFDRWcMCi9BGEh54scjR3Pkr3WbWRatVEsgSobijODgeU25T7C7pmufzjqCQJ+h2GMySoeMitd9rmCBhTCKM2Ml72WaMwl+B+WA1CXvj1okpS23RFIw== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1721155066; bh=A3ZU19SF5dlrRJD5L1eQdZQZ3e9JK8z+mGxqvtrcXnH=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=O+13bLRIBr3HQjEEXmSC8+Ed47iPZCmV1hd6et1q8I3bL6d6dnWTVnFsQLTJQTpP85jcmuYxusd8vn+di23zwosl9T3mEzXPOjuxg1OBrcFIS7pvjaaVvQ5tQcttPwdayzTbjbpA9LAMPxTRoU9tt2TzY4O2sEi0QMBWLEds2GDjHcLHFMnwQoDpkk3uE3LhLgY0DqDsxfUPhK+cPSp0Kin9aR10ZWBYzoCD/C5XlrYnFlQL2+zNhLw6EWrkjgoRvQj8FkILzFtn7T/cW7v1PU+7QKDT4O2jRHAF8VCHcRu+RxcuarV042+I5E9mJsXvC+amo20erHw2LIHOE5iYFw== X-YMail-OSG: HEeEAToVM1lhY7qLTOCm3eNtEYjRpop4xCafN.L0z7DPIJOFrrL6IzhLUzGWorF AdyiWs6d41d5pOc4q7Pazrn5S.iyzWKHTUuyFrikbHFRWJM7SYDGi_vqdzmnFOvUDLAcMVy9Xosj zM9h4521qO4qqLcfmjWqSLfFrWUz6QLmUaVIewQVPJh5IrK3MaQEhW6Ebz6GOGFGEcGrBjJif_A2 MSg3xmiqhEPmNabs2GYvvn5io.1o5gcVWnFAw06R4MV3ULIwmXivYZg9CeUN5p2NVJSovtj7WYcY if6JYiEuM8PHlEBKqvTymjYGGXZlbn0YiG._xpia9bdHL66wcPR0Z1RIgpZA5wcd52Qb1OfwK_g_ 39cTaeGNjgepbb9fDUTmnp004Hk_n1t_RkMgjiZBr2fxvrdE6S7XwqrNyqcJInYmErnuk.wmJpjC 8GW.9wDEMTyfmOIiOs6GNVudHy_A4QCX10T8379GLJBekz_c2sndxO0rnnVZyQkecP.Vz5fVTNqV tnOcRCIvYfdQpaDkBnNFf156dEtFCnaPiw8.CZOTfLFxOstRCPdgachIB_ygwEPD2MNCaEUqjx4W 9Lo11DraDXLgwZOl53uTIW0DRBav60J1RdtjOD0BPR5VgzeYAYGW0P0mxAwTckCY9rJSb4YpuMsg 4lAg140AmB1S0M5fhahHIxp4ojRepYeGbbRahs1.AaZiMJoDUbI0xq0NXQjKQdHNFSJ0R4KRCfjm FtHC5ewfHz7qv_JQGqPSA_4rph3KGJBpUSys97cwJc8DDG8dbmucsfX0eIdIMHvlK_Eaj9FKgKv2 Me7UBfKjs6tVJEVfi_9ZutVZszfH9BPptmO7FPD8WFMt5E9glO561ELlkisFIZ490CY7r8ZuRqrr WUHJMNd_x__mpDRTUm1XtL.njHWIlDB5PWtsKyExYh_VKk1rfQ5yZ0BrmwdFgYUx51roAwxR4Fvu XPwqxQIQt.SF5bj8TZvtRg6MpEo87zmd93mmiQci_amrKSxQ8HvlQmv.CQ1gGbuoUCXR5si.tO52 rtaQ6I0HH1n5iza0P9tjAY2dCWeEdpSwgKiVHA7ZQaVof_buhYOasq1Kb4oWCUZSm.UAK8NOub5Y KwI1bq_yBgosBT64HOoQ.lKxdu10Zkv3CdaNDFG7mDqEYybkXOaZxgN18gV6YYDPFbFxqu63MB8y 9id4rjz4w2SCqix7LEa5u2rnhSgZBseTUtXnueNZKUV0TSdF419vkueDfelikQbwSBZA_hXfa3jo TKEDwOyFLA8Fe09uUPp2L_EAdO77KM_d6xiG8SIzFxJaePo5fYsgZopfncbMx2.A55k.Q4FwyC7V gOfakQ94_VR_VTxXZ_bhY27Vgx.uLEW1gJ.nR1Ab7CNl7b1hHyWTxc81n0a50azknUt7SCLmhtX9 Cl7yY3H6fJmzy75YKTHMximMFPsroCNcNaikrWHRmGmd6ovgm5PmmvVb_4RP5fmKixHZrZ_VOSse vLnRFjXWPLi_exhSweoJHCkPxOjUDLW.PC9og3xGCxBLt5M7HooX7e2gosDfnobqdnHxSqAeU1kK wvZ6Ct8sv4.steuuaNr8eN1Xr2EmyyWhqSWWRB1mrGVIwxdYy5GhPu8RmwV7246sSY6BMV0QH.Yf GXwPd1BIMXG0PcKypij4ciwcg4_9.jtQUTr0HIP._DDpbulK8T8pl08uafPyJsxPG1ANyWTk0o5M JLuXfMQYrOSAqlvS0YmKfKSuygu4SFPlpJjztkVNmw6imDfzeGaUJElUOV7Iv.KYf3lZWiRX6hMz FrccguFtzjgvqs6x4DYEw1.wPMTpyVe2en_EdL0Lnnt3seSQv6r7V_EMorExRK26W1C3aRZLLv5Y 6KOwSlWGrMCFxQq3r0vqg0FzSsh2Bi_xQcRTixIT7GUyj4FmGSW8xgDEkUP0.0ul2LUCuM_SlYSA doO9UMGp2pdMcfSMELKc.EUhKQVy9o5O.Mss84IOtwwB4rIF_uXl8L38ha3kyIc_2QXOE2xYY9sK o645aD2sWaxmHEIeGLNsG.nCgEiUpzdMatw9VjtZhBu1tll1odKP8BsXnxHw792EGyD7h7hApZ1j DiiWNVS6J3Llj.h1aEnqctZW9y5215t0SWjJ7RhSAlYVMFUum.RPfjs.5y40QnZtrLqd.QZckbmc ZaseIWnQin1ITqP5jsG5QGOcQ_T0HUMPjQM59UlXCotuiAgm1PemFwsbc7ONvlBmL1knsIAXDwkY X82pzyPgb4dZT66Vvai6k92An2hzQ_iCH7ewIk1ONQfsaJfOH6eGa_8svaszQ_ff57vyTdaDvUnU 1bGLtcz4R X-Sonic-MF: X-Sonic-ID: fe07fe3d-56cb-475e-a0bf-0f6ef90cce92 Received: from sonic.gate.mail.ne1.yahoo.com by sonic317.consmr.mail.gq1.yahoo.com with HTTP; Tue, 16 Jul 2024 18:37:46 +0000 Received: by hermes--production-gq1-799bb7c8cf-hxpdl (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 199a87725a74327994cb147b0ed289f7; Tue, 16 Jul 2024 18:37:45 +0000 (UTC) Content-Type: text/plain; charset=utf-8 List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.600.62\)) Subject: Re: armv7-on-aarch64 stuck at urdlck: I got a replication of the "ampere2" bulk build hangup problem on a Windows DevKit 2023 From: Mark Millard In-Reply-To: Date: Tue, 16 Jul 2024 11:37:34 -0700 Cc: Konstantin Belousov , Baptiste Daroussin Content-Transfer-Encoding: quoted-printable Message-Id: <8214703E-AB28-4FB3-A3DD-03C87363D8C6@yahoo.com> References: To: "olce@freebsd.org" , Philip Paeps , FreeBSD ARM List , FreeBSD Mailing List , Current FreeBSD X-Mailer: Apple Mail (2.3774.600.62) X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.93 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.93)[-0.927]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; DKIM_TRACE(0.00)[yahoo.com:+]; FREEMAIL_FROM(0.00)[yahoo.com]; TO_DN_SOME(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MLMMJ_DEST(0.00)[freebsd-arm@freebsd.org]; RCPT_COUNT_SEVEN(0.00)[7]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.66.146:from]; APPLE_MAILER_COMMON(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.66.146:from] X-Rspamd-Queue-Id: 4WNnqy1C4wz4r3R On Jul 16, 2024, at 10:42, Mark Millard wrote: > No longer is the problem only observed on ampere2! But this was with > a non-debug, personally built kernel that has some of my now patches. > I'll see if I can replicate the issue with an official pkgbase debug > kernel. It replicated with the official pkgbase debug kernel. The kernel did not report anything. The following commits in main and happen between the last working ampere2 armv7 builds and the first failing ampere2 builds and look be the only likely contributors from that range as far as I could tell: Tue, 27 Feb 2024 . . . =E2=80=A2 git: 1df8700aa6cf - main - PP mutexes: unlock: Reset inherited = prio regardless of privileges Olivier Certner=20 =E2=80=A2 git: 9ac3ac9ece62 - main - PP mutexes: lock: Check if priority = is too high against base one Olivier Certner=20 =E2=80=A2 git: 39e4665c9694 - main - PP mutexes: lock: Reduce = 'umtx_lock' holding before taking the user lock Olivier Certner These changes are not in 14.0-RELEASE but are in 14.1-STABLE and = 14.1-RELEASE. So I expect that when any ampere*'s progress to 14.1-RELEASE the armv7 problems would start for them. These changes are not in 13.3-RELEASE but are in 13.3-STABLE. So I = expect that when any ampere*'s progress to 13.4-RELEASE the problems would start for = them. With the prior packages already built in a prior poudriere-devel run it turns out that just: # poudriere bulk -j main-armv7-poud -i graphics/graphviz replicates the problem: . . . [00:00:45] Installing graphics/graphviz | graphviz-9.0.0_4 [aarch64PBase] Installing graphviz-9.0.0_4... [aarch64PBase] `-- Installing cairo-1.17.4_2,3... [aarch64PBase] | `-- Installing fontconfig-2.15.0_2,1... [aarch64PBase] | | `-- Installing expat-2.6.2... [aarch64PBase] | | `-- Extracting expat-2.6.2: 100% [aarch64PBase] | | `-- Installing freetype2-2.13.2... [aarch64PBase] | | `-- Installing brotli-1.1.0,1... [aarch64PBase] | | `-- Extracting brotli-1.1.0,1: 100% [aarch64PBase] | | `-- Installing png-1.6.43... [aarch64PBase] | | `-- Extracting png-1.6.43: 100% [aarch64PBase] | | `-- Extracting freetype2-2.13.2: 100% [aarch64PBase] | `-- Extracting fontconfig-2.15.0_2,1: 100% [aarch64PBase] | `-- Installing glib-2.80.4,2... [aarch64PBase] | | `-- Installing libffi-3.4.6... [aarch64PBase] | | `-- Extracting libffi-3.4.6: 100% [aarch64PBase] | | `-- Installing libiconv-1.17_1... [aarch64PBase] | | `-- Extracting libiconv-1.17_1: 100% [aarch64PBase] | | `-- Installing pcre2-10.43... [aarch64PBase] | | `-- Extracting pcre2-10.43: 100% [aarch64PBase] | | `-- Installing py311-packaging-24.1... [aarch64PBase] | | `-- Installing python311-3.11.9_1... [aarch64PBase] | | | `-- Installing mpdecimal-4.0.0... [aarch64PBase] | | | `-- Extracting mpdecimal-4.0.0: 100% [aarch64PBase] | | | `-- Installing readline-8.2.10... [aarch64PBase] | | | `-- Extracting readline-8.2.10: 100% [aarch64PBase] | | `-- Extracting python311-3.11.9_1: 100% [aarch64PBase] | | `-- Extracting py311-packaging-24.1: 100% [aarch64PBase] | `-- Extracting glib-2.80.4,2: 100% [aarch64PBase] | `-- Installing libglvnd-1.7.0... [aarch64PBase] | `-- Extracting libglvnd-1.7.0: 100% [aarch64PBase] | `-- Installing pixman-0.42.2... [aarch64PBase] | `-- Extracting pixman-0.42.2: 100% [aarch64PBase] `-- Extracting cairo-1.17.4_2,3: 100% [aarch64PBase] `-- Installing harfbuzz-9.0.0... [aarch64PBase] | `-- Installing graphite2-1.3.14... [aarch64PBase] | `-- Extracting graphite2-1.3.14: 100% [aarch64PBase] `-- Extracting harfbuzz-9.0.0: 100% [aarch64PBase] `-- Installing jpeg-turbo-3.0.3... [aarch64PBase] `-- Extracting jpeg-turbo-3.0.3: 100% [aarch64PBase] `-- Installing libgd-2.3.3_13,1... [aarch64PBase] | `-- Installing tiff-4.6.0... [aarch64PBase] | | `-- Installing jbigkit-2.1_3... [aarch64PBase] | | `-- Extracting jbigkit-2.1_3: 100% [aarch64PBase] | | `-- Installing lerc-4.0.0... [aarch64PBase] | | `-- Extracting lerc-4.0.0: 100% [aarch64PBase] | | `-- Installing libdeflate-1.20... [aarch64PBase] | | `-- Extracting libdeflate-1.20: 100% [aarch64PBase] | | `-- Installing zstd-1.5.6... [aarch64PBase] | | `-- Installing liblz4-1.9.4_1,1... [aarch64PBase] | | `-- Extracting liblz4-1.9.4_1,1: 100% [aarch64PBase] | | `-- Extracting zstd-1.5.6: 100% [aarch64PBase] | `-- Extracting tiff-4.6.0: 100% [aarch64PBase] | `-- Installing webp-1.4.0_1... [aarch64PBase] | | `-- Installing giflib-5.2.2... [aarch64PBase] | | `-- Extracting giflib-5.2.2: 100% [aarch64PBase] | `-- Extracting webp-1.4.0_1: 100% [aarch64PBase] `-- Extracting libgd-2.3.3_13,1: 100% [aarch64PBase] `-- Installing libltdl-2.4.7... [aarch64PBase] `-- Extracting libltdl-2.4.7: 100% [aarch64PBase] `-- Installing pango-1.52.2_1... [aarch64PBase] | `-- Installing fribidi-1.0.15... [aarch64PBase] | `-- Extracting fribidi-1.0.15: 100% [aarch64PBase] | `-- Installing libXft-2.3.8... [aarch64PBase] | `-- Extracting libXft-2.3.8: 100% [aarch64PBase] | `-- Installing libthai-0.1.29_1... [aarch64PBase] | | `-- Installing libdatrie-0.2.13_2... [aarch64PBase] | | `-- Extracting libdatrie-0.2.13_2: 100% [aarch64PBase] | `-- Extracting libthai-0.1.29_1: 100% [aarch64PBase] `-- Extracting pango-1.52.2_1: 100% [aarch64PBase] Extracting graphviz-9.0.0_4: 100% And here it is hung with /usr/local/bin/dot -c in urdlck : 0 1483 4502 7 68 0 15760 4872 wait I+ 0 0:03.92 | | = `-- /usr/local/libexec/poudriere/sh -e -o pipefail = /usr/local/share/poudriere/bulk.sh -j main-armv7-poud -i = graphics/graphviz 0 1894 1483 5 68 0 15760 4712 nanslp S 0 0:02.07 | | = |-- sh: poudriere[main-armv7-poud-default]: html_json_main (sh) 0 25321 1483 6 68 0 6664 3868 wait I+J 0 0:00.11 | | = `-- /usr/bin/make -C /usr/ports/graphics/graphviz install-package 0 25322 25321 5 68 0 11140 8860 wait I+J 0 0:00.00 | | = `-- /usr/local/sbin/pkg-static add /packages/All/graphviz-9.0.0_4.pkg 0 25323 25322 5 20 0 63824 45144 select S+J 0 0:02.85 | | = `-- /usr/local/sbin/pkg-static add = /packages/All/graphviz-9.0.0_4.pkg 0 26900 25323 3 68 0 26292 23804 urdlck I+J 0 0:00.02 | | = `-- /usr/local/bin/dot -c > FYI for the replication that I got: >=20 > /usr/local/sbin/pkg-static add -A /packages/All/graphviz-9.0.0_4.pkg > runs: > /usr/local/bin/dot -c >=20 > each such /usr/local/bin/dot is stuck at MWCHAN urdlck . >=20 > # poudriere status -b > [main-armv7-poud-default] [2024-07-16_04h27m31s] [parallel_build] = Queued: 449 Built: 433 Failed: 0 Skipped: 0 Ignored: 0 Fetched: 0 = Tobuild: 16 Time: 04:44:38 > ID TOTAL ORIGIN PKGNAME = PHASE PHASE TMPFS CPU% MEM% > [01] 00:59:45 graphics/rubygem-ruby-graphviz | = rubygem-ruby-graphviz-1.2.5 run-depends 00:59:30 1.59 GiB 0% 0.2% > [02] 00:49:59 graphics/p5-GraphViz | p5-GraphViz-2.25 = build-depends 00:49:53 1.45 GiB 0% 0.2% > [03] 00:59:45 graphics/py-pydot@py311 | py311-pydot-2.0.0 = run-depends 00:59:34 1.47 GiB 0% 0.2% > [04] 00:59:45 graphics/py-pygraphviz@py311 | py311-pygraphviz-1.6 = lib-depends 00:59:33 1.47 GiB 0% 0.2% > [05] 00:58:57 graphics/py-graphviz@py311 | py311-graphviz-0.10.1 = run-depends 00:58:49 1.47 GiB 0% 0.2% > [06] 00:59:20 audio/ganv | ganv-1.8.2_1 = lib-depends 00:59:10 1.53 GiB 0% 0.2% > [07] 00:59:45 devel/libr3 | libr3-1.0.0_2 = lib-depends 00:59:19 1.54 GiB 0% 0.3% > [08] 00:59:45 net/netmap | netmap-0.1.3_2 = run-depends 00:59:22 1.46 GiB 0% 0.3% >=20 > I had started the bulk build via the list: >=20 > # more ~/origins/ampere2-failures-armv7.txt > audio/ganv > devel/doxygen > devel/libr3 > graphics/p5-GraphViz > graphics/p5-GraphViz2 > graphics/oyranos > graphics/pear-Image_GraphViz@php81 > graphics/py-graphviz@py311 > graphics/py-pydot@py311 > graphics/py-pygraphviz@py311 > graphics/rubygem-ruby-graphviz > math/ggobi > net-mgmt/librenms > net/netmap > print/dot2tex@py311 >=20 > # poudriere bulk -j main-armv7-poud `cat = ~/origins/ampere2-failures-armv7.txt` > . . . > [00:00:12] Building 449 packages using up to 8 builders > . . . > [03:44:55] [01] [00:18:54] Finished graphics/graphviz | = graphviz-9.0.0_4: Success > [03:44:56] [01] [00:00:00] Building graphics/rubygem-ruby-graphviz | = rubygem-ruby-graphviz-1.2.5 > [03:44:56] [03] [00:00:00] Building graphics/py-pydot@py311 | = py311-pydot-2.0.0 > [03:44:56] [04] [00:00:00] Building graphics/py-pygraphviz@py311 | = py311-pygraphviz-1.6 > [03:44:56] [07] [00:00:00] Building devel/libr3 | libr3-1.0.0_2 > [03:44:56] [08] [00:00:00] Building net/netmap | netmap-0.1.3_2 > [03:45:21] [06] [00:09:33] Finished x11-toolkits/gtkmm24 | = gtkmm24-2.24.5_4: Success > [03:45:21] [06] [00:00:00] Building audio/ganv | ganv-1.8.2_1 > [03:45:44] [05] [00:13:33] Finished graphics/ImageMagick6@nox11 | = ImageMagick6-nox11-6.9.12.77_9,1: Success > [03:45:44] [05] [00:00:00] Building graphics/py-graphviz@py311 | = py311-graphviz-0.10.1 > [03:54:42] [02] [00:24:53] Finished print/texlive-base | = texlive-base-20240312: Success > [03:54:42] [02] [00:00:00] Building graphics/p5-GraphViz | = p5-GraphViz-2.25 >=20 > In /usr/src/sys/kern/kern_umtx.c there is: >=20 > static int > do_rw_rdlock(struct thread *td, struct urwlock *rwlock, long fflag, > struct _umtx_time *timeout) > { > . . . > /* > * Contention bit is set, before sleeping, increase > * read waiter count. > */ > rv =3D fueword32(&rwlock->rw_blocked_readers, > &blocked_readers); > if (rv =3D=3D 0) > rv =3D suword32(&rwlock->rw_blocked_readers, > blocked_readers + 1); > if (rv =3D=3D -1) { > umtxq_unbusy_unlocked(&uq->uq_key); > error =3D EFAULT; > break; > } > while (state & wrflags) { > umtxq_lock(&uq->uq_key); > umtxq_insert(uq); > umtxq_unbusy(&uq->uq_key); > error =3D umtxq_sleep(uq, "urdlck", timeout = =3D=3D NULL ? > NULL : &timo); > umtxq_busy(&uq->uq_key); > umtxq_remove(uq); > umtxq_unlock(&uq->uq_key); > if (error) > break; > rv =3D fueword32(&rwlock->rw_state, &state); > if (rv =3D=3D -1) { > error =3D EFAULT; > break; > } > } >=20 > . . . >=20 >=20 >=20 > For reference: >=20 > # ps -alxdww | less > UID PID PPID C PRI NI VSZ RSS MWCHAN STAT TT TIME = COMMAND > . . . > 0 87700 4522 6 20 0 16576 1888 - T 0 0:00.01 | | = |-- vi /usr/local/share/poudriere/jail.sh > 0 91496 4522 4 20 0 15760 4684 select S+ 0 0:06.88 | | = `-- /usr/local/libexec/poudriere/sh -e -o pipefail = /usr/local/share/poudriere/bulk.sh audio/ganv devel/doxygen devel/libr3 = graphics/p5-GraphViz graphics/p5-GraphViz2 graphics/oyranos = graphics/pear-Image_GraphViz@php81 graphics/py-graphviz@py311 = graphics/py-pydot@py311 graphics/py-pygraphviz@py311 = graphics/rubygem-ruby-graphviz math/ggobi net-mgmt/librenms net/netmap = print/dot2tex@py311 > 0 37688 91496 0 68 0 15760 4700 wait I 0 0:00.05 | | = |-- sh: poudriere[main-armv7-poud-default][01]: build_pkg = (rubygem-ruby-graphviz-1.2.5) (sh) > 0 47568 37688 0 68 0 6664 3664 wait IJ 0 0:00.03 | | = | `-- /usr/bin/make -C /usr/ports/graphics/rubygem-ruby-graphviz = run-depends > 0 47598 47568 6 68 0 5568 2988 wait IJ 0 0:00.01 | | = | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh > 0 47743 47598 6 68 0 11188 8864 wait IJ 0 0:00.00 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg > 0 47747 47743 3 20 0 71692 48984 select SJ 0 0:04.26 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg > 0 56383 47747 1 68 0 26292 23812 urdlck IJ 0 0:00.02 | | = | `-- /usr/local/bin/dot -c > 0 37700 91496 6 68 0 15760 4700 wait I 0 0:00.04 | | = |-- sh: poudriere[main-armv7-poud-default][03]: build_pkg = (py311-pydot-2.0.0) (sh) > 0 45102 37700 2 68 0 6668 3704 wait IJ 0 0:00.02 | | = | `-- /usr/bin/make -C /usr/ports/graphics/py-pydot FLAVOR=3Dpy311 = run-depends > 0 45156 45102 4 68 0 5584 2992 wait IJ 0 0:00.01 | | = | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh > 0 45215 45156 4 68 0 11144 8864 wait IJ 0 0:00.00 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg > 0 45218 45215 4 20 0 51420 31512 select SJ 0 0:02.68 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg > 0 52147 45218 2 68 0 26292 23812 urdlck IJ 0 0:00.02 | | = | `-- /usr/local/bin/dot -c > 0 37721 91496 1 68 0 15760 4700 wait I 0 0:00.04 | | = |-- sh: poudriere[main-armv7-poud-default][04]: build_pkg = (py311-pygraphviz-1.6) (sh) > 0 45937 37721 1 68 0 6684 3744 wait IJ 0 0:00.03 | | = | `-- /usr/bin/make -C /usr/ports/graphics/py-pygraphviz FLAVOR=3Dpy311= lib-depends > 0 46009 45937 7 68 0 5584 2992 wait IJ 0 0:00.01 | | = | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh > 0 46127 46009 7 68 0 11144 8864 wait IJ 0 0:00.00 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg > 0 46129 46127 1 20 0 51384 31548 select SJ 0 0:02.73 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg > 0 53311 46129 4 68 0 26292 23812 urdlck IJ 0 0:00.02 | | = | `-- /usr/local/bin/dot -c > 0 37744 91496 7 45 0 15760 4692 wait I 0 0:00.04 | | = |-- sh: poudriere[main-armv7-poud-default][07]: build_pkg = (libr3-1.0.0_2) (sh) > 0 55198 37744 0 50 0 6664 3664 wait IJ 0 0:00.03 | | = | `-- /usr/bin/make -C /usr/ports/devel/libr3 lib-depends > 0 55229 55198 0 68 0 5588 2988 wait IJ 0 0:00.01 | | = | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh > 0 55594 55229 7 68 0 11168 8864 wait IJ 0 0:00.00 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg > 0 55596 55594 2 20 0 69796 50180 select SJ 0 0:04.53 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg > 0 62753 55596 0 68 0 26292 23812 urdlck IJ 0 0:00.02 | | = | `-- /usr/local/bin/dot -c > 0 37763 91496 4 29 0 15760 4696 wait I 0 0:00.05 | | = |-- sh: poudriere[main-armv7-poud-default][08]: build_pkg = (netmap-0.1.3_2) (sh) > 0 51054 37763 6 36 0 6636 3684 wait IJ 0 0:00.03 | | = | `-- /usr/bin/make -C /usr/ports/net/netmap run-depends > 0 51107 51054 3 68 0 5568 2988 wait IJ 0 0:00.01 | | = | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh > 0 51576 51107 3 68 0 11168 8860 wait IJ 0 0:00.00 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg > 0 51580 51576 3 20 0 68220 49432 select SJ 0 0:04.27 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg > 0 59063 51580 6 68 0 26292 23808 urdlck IJ 0 0:00.02 | | = | `-- /usr/local/bin/dot -c > 0 53709 91496 1 68 0 15760 4700 wait I 0 0:00.04 | | = |-- sh: poudriere[main-armv7-poud-default][06]: build_pkg = (ganv-1.8.2_1) (sh) > 0 63371 53709 6 68 0 6636 3668 wait IJ 0 0:00.03 | | = | `-- /usr/bin/make -C /usr/ports/audio/ganv lib-depends > 0 63377 63371 1 68 0 5580 2996 wait IJ 0 0:00.01 | | = | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh > 0 63413 63377 4 68 0 11180 8864 wait IJ 0 0:00.00 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg > 0 63414 63413 3 20 0 56212 35980 select SJ 0 0:02.21 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg > 0 65577 63414 6 68 0 26292 23812 urdlck IJ 0 0:00.01 | | = | `-- /usr/local/bin/dot -c > 0 63365 91496 3 68 0 15760 4696 wait I 0 0:00.03 | | = |-- sh: poudriere[main-armv7-poud-default][02]: build_pkg = (p5-GraphViz-2.25) (sh) > 0 63807 63365 7 68 0 6696 3672 wait IJ 0 0:00.02 | | = | `-- /usr/bin/make -C /usr/ports/graphics/p5-GraphViz build-depends > 0 63808 63807 2 68 0 5568 2988 wait IJ 0 0:00.01 | | = | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh > 0 63833 63808 4 68 0 11188 8864 wait IJ 0 0:00.00 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg > 0 63834 63833 7 20 0 67400 48532 select SJ 0 0:03.52 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg > 0 66041 63834 2 68 0 26292 23812 urdlck IJ 0 0:00.02 | | = | `-- /usr/local/bin/dot -c > 0 69974 91496 1 68 0 15760 4700 wait I 0 0:00.04 | | = |-- sh: poudriere[main-armv7-poud-default][05]: build_pkg = (py311-graphviz-0.10.1) (sh) > 0 73474 69974 5 68 0 6684 3740 wait IJ 0 0:00.02 | | = | `-- /usr/bin/make -C /usr/ports/graphics/py-graphviz FLAVOR=3Dpy311 = run-depends > 0 73496 73474 6 68 0 5584 2992 wait IJ 0 0:00.01 | | = | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh > 0 73521 73496 7 68 0 11144 8864 wait IJ 0 0:00.00 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg > 0 73522 73521 0 20 0 52432 32664 select SJ 0 0:02.70 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg > 0 76540 73522 3 68 0 26292 23812 urdlck IJ 0 0:00.02 | | = | `-- /usr/local/bin/dot -c > 0 91907 91496 5 68 0 15760 4492 nanslp S 0 1:05.17 | | = |-- sh: poudriere[main-armv7-poud-default]: html_json_main (sh) > 0 99134 91496 1 40 0 15760 4740 piperd I 0 0:03.22 | | = `-- sh: poudriere[main-armv7-poud-default]: pkg_cacher_main (sh) > 0 23 =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Wed Jul 17 01:41:44 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WNzFP0cPfz5QhJ1 for ; Wed, 17 Jul 2024 01:42:01 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic316-55.consmr.mail.gq1.yahoo.com (sonic316-55.consmr.mail.gq1.yahoo.com [98.137.69.31]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4WNzFM6Gwvz4h5w for ; Wed, 17 Jul 2024 01:41:59 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=rPNLq7g0; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.69.31 as permitted sender) smtp.mailfrom=marklmi@yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1721180515; bh=Ux8nkaFfLEyjLspKezqFB2PUcSk2kfFnvRyzsdacZr4=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=rPNLq7g0ZfAZiGcJ/uC7JDDMt+EZkfTHAHPCPPI3vfIwW+jH3yLZBstNLiVSPolxLwcPa9own4p/zBwbqMAYxf8KM5TOrdLfEErFUmOhzA5se7G9wuHTulvEqfLTWj6gy2+Ncj0j9JvAjaQZCeys3DhisTStIrKKf9OnG5yax2tKaQsy0l1PlRPuEcMAiAaZW14zcI6qQFvok1hEp2LrsMDjwD9C67l+YWqYMAMhFIimV71zwygKH0dPV4QP1Cu9Vq4Gs64ATRtHbMcqg+eP/TwYDtjrkJP5Qn6hIhlbLP9EA/De8h8DWCJx5QoBQLjaXOr8PJQP8A9dTPvshSQV3A== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1721180515; bh=3XPK/fJq18we8grTCEHV9cPIX1pTbBXy+Q09FRAst/x=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=Pls+fcb8dH3r/OOt8JfhNEkz6yIiGN7PmO91l/XAZ+Jf+URTiGffAyFyq+mlWE4aySzy2Ou2R0l9ubwx5wOrs8nf0hiRoPCjel2Iw+G8vAnOJ67MUz8EppaeSj/sgMA06/Qr5I7IQYX9I+F79rvU7erzfSUsYyReB3ZkbD+L4JEG6ffmEHRf+fOxqNelAWEquu45s/FxmU+jMIi2HWLGU3LKNIzZpLdvLKMF6xVympNRns+9l7hTEc8DKoJwZUHA6mcfG84QjDCfKc2B2oLdQQhWig+tw8eRMYFygzTRHyof5sSYcd1craZaC/e8sFOmadHhNNKHSk+Swz7DkGzKdg== X-YMail-OSG: eMfJEJMVM1lTh0xBF5I_6k4kEv_Y.fTC7Ccw.V0VeAnadTAwlvgdlN8wYMXZxFE sqtfozuHWaL0x938jrbpxYAqCJAY1xexh5ByziKoNetPNxKrUiS1.Dq9a78t9nfkZOW6DRB_cWlE 8tabfTDtSPMGwYIqsC2y9d3BNuuOH3jUt.gOHtBil3QvyPiSnjh3YeR.xBwRjgbYDsHJB.LAK93i pKX531fO9d7gEF7FYmE3oTyIZ2N6SufnZBUknuiazPy6.de_gKgmvfVggGlllRt6JGSIjsg5m9Fk ol.Uin131NixMCNoBG3WWnAwOn0YJmaSp0M761hta90JnFOPsOeKcGVJyPhfmuLFYZAXG0zMrek. 9DDKbPWo5JkQ5Kg4TZSN_Oml1qR3LQobe_VhF.EcBnLDLQu64V7p2xxDVtGlW5Zj4Yz6ai3EYwXM p.qTz9XwldxqEBpkcJDiFKNDdaVLeU7bxX_3lEtaKCeW_OvpR.PlxQvWEIbFGu6P.oRI3JCdJOE7 2.2ZG4w65CdpQDnVSacJpW1Bd9dyBuzLb6lTDcFnvikIBZ8LNEqEWtxeHHLUhIZfzRsopraXc5CQ _.HC_moOrLU9TjSv0lqDBmR4o5Uof5bWBfQtaRu0Fg2_gpsdNO3426TLKrbdVGKfKUzKv6LLJ5cO KH98aKASJbCZSHBnCsIyVHx.qalZx6F5GYbLhOdpeOoXd0CsgvSSGWq..fkXjnSSa08.EFL427dp Z.KhAUAiHdBMm87yWvMeoimKGBxSXsPUoAWxpdzGffEbSGvkMFMKx8Mm69N.rBr8xQnMWbtllzAm YfrW9Ys4Yrg_hu0o9lUEwhm6SC4y2j9sUaq8RlYY5AJPigTjQlud4nfhiOBOszyPeUTcQpN1.0wx nHSU9fiN8dRxfx7L4cB8I_VeIx_unloALf05hLJLV.PcI3yd3iH323u4.TQPFv7DBd1O12CX6It_ IwFEz.R4Rcib7EKf6fOVCDz3q6JSo3EFFpF5_gy7nSywCM_jqC17KNc2Q7kNaUIf09w70zvJ5TUe Fy.oXcsMbJCBDptwTsrX94soxKdmu7LVXWeD2INtuQpSU9nR7v8ltqUJ676WqoOB8yWCVcvcbuIV 0W5G4eQ1M7pkdCmnIvLWvrXHk0GP0r5n4_TtiNaobNSzlCjK.P1cJl3Y06mxwOqVxZCcwrS0rKNi BIOflnMrNRcQm6OhsdC6v1FUkkVVf.pSs3Q56cE93mb0_MPifF2556yiw3CNIRSZnFomTmKhy7XS e_G_niWyCUx5YZ7qU4s62NCArQ45AS9nM_Ih8NBE31SMcuxzxZ3QIrWZKaJotS34gJxUYIwEzaxQ U8OkuhD8tj22r1VfBeCMJemhXfYuuo7eBBoX5JnrQrmYe1jIL.d_srCotB0zLXG94.oM2BcLyghb SB.0X7LenfkxJTtJvsOl8kxRv116LZFet3sYccDx6RpfiMLU63EyuhJrutKqkGDK974qKVW.0Ycd 7qiXoB0A8cEJy45611pxyQfJ1OMnS7xUm04d2JasozTqQ5S6fyhEZdrfUrXHoZFb__OGBgjdFHup UwekfL49fNVUXVfjfmJXSEKqC6hL43OdrJ0eEgU0BxSMjDWbEKuiPoSXbbTh76WYa53aVd4iixF_ NdFswYRTZBET8VwwvX7i0QGU95eyc5DN7cLX_oSRgtKuED8KFIkaS1GxnOOKlW9ttk_ApHHVrrTi CdKA_FuCv1eqpd_kyr6pIFP6_wd.NDgPtwRQwRLT2cb5kXLjvQHJ0dPBGx2KbBJ97XcPkcg_QqOZ FPbInBYzxBB3Z3IjvIaO21g.Q_f.3C9nl5MdXLsf99D5Y49_zxec7zkfP3cJMQKM3uSri4_riTp7 iQuy5ijy6mY3be.BK_0l3aQMDUa4N.mjPHGsVVZq07cxT_QG.ViOclg.fmxYLw37i3tQ.LK2Ykfw UG_Ho.JCf_Mq6aWjyMNXUEJKILkOrRBdTkzH.EybzXlBY8gCsD7o3yolGx3r6vZnNQ7pCFpNOFpr iPUBza8LGn6wOeUL6ZGuHYw1TBJ65FtZo.h9Tc3i61nMCdRKK64yg_hsdKXiHKA6vLX.MJ11TQma ueDFz2lbm9LOBBzJI_lvWS4vnSNC75IcSOT2aSZwaB6TzMIqkqlw_CSiqrGwOpmkLQOPjR311lt1 JZ.NjkeJx32LQEXcEO9oa3rk3fcQ0t0W4ICuQiaOInlFbb94L72DCz4fNVjxOvcYsma6sLfZUUBP O.KdZ.CcD_Mp7QPZhsE.XpSmJBpgtlZrpv2XNb2pSBTK1f8HkJOg8f6NbqGxkk_dvld4fl9oM9zw U1CGPsYJKZJ2c X-Sonic-MF: X-Sonic-ID: 41d345e8-7b5a-42bb-83ab-c73e9066a13b Received: from sonic.gate.mail.ne1.yahoo.com by sonic316.consmr.mail.gq1.yahoo.com with HTTP; Wed, 17 Jul 2024 01:41:55 +0000 Received: by hermes--production-gq1-799bb7c8cf-t9jf4 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 50acf04bab00e7b494e2f560782adad9; Wed, 17 Jul 2024 01:41:54 +0000 (UTC) Content-Type: text/plain; charset=utf-8 List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.600.62\)) Subject: Re: armv7-on-aarch64 stuck at urdlck: I got a replication of the "ampere2" bulk build hangup problem on a Windows DevKit 2023 From: Mark Millard In-Reply-To: <8214703E-AB28-4FB3-A3DD-03C87363D8C6@yahoo.com> Date: Tue, 16 Jul 2024 18:41:44 -0700 Cc: Konstantin Belousov , Baptiste Daroussin Content-Transfer-Encoding: quoted-printable Message-Id: References: <8214703E-AB28-4FB3-A3DD-03C87363D8C6@yahoo.com> To: "olce@freebsd.org" , Philip Paeps , FreeBSD ARM List , FreeBSD Mailing List , Current FreeBSD X-Mailer: Apple Mail (2.3774.600.62) X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.84 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.84)[-0.837]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; DKIM_TRACE(0.00)[yahoo.com:+]; FREEMAIL_FROM(0.00)[yahoo.com]; TO_DN_SOME(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MLMMJ_DEST(0.00)[freebsd-arm@freebsd.org]; RCPT_COUNT_SEVEN(0.00)[7]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.31:from]; APPLE_MAILER_COMMON(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.31:from] X-Rspamd-Queue-Id: 4WNzFM6Gwvz4h5w On Jul 16, 2024, at 11:37, Mark Millard wrote: > On Jul 16, 2024, at 10:42, Mark Millard wrote: >=20 >> No longer is the problem only observed on ampere2! But this was with >> a non-debug, personally built kernel that has some of my now patches. >> I'll see if I can replicate the issue with an official pkgbase debug >> kernel. >=20 > It replicated with the official pkgbase debug kernel. The > kernel did not report anything. >=20 > The following commits in main and happen between the last working > ampere2 armv7 builds and the first failing ampere2 builds and > look be the only likely contributors from that range as far as > I could tell: >=20 > Tue, 27 Feb 2024 > . . . > =E2=80=A2 git: 1df8700aa6cf - main - PP mutexes: unlock: Reset = inherited prio regardless of privileges Olivier Certner=20 > =E2=80=A2 git: 9ac3ac9ece62 - main - PP mutexes: lock: Check if = priority is too high against base one Olivier Certner=20 > =E2=80=A2 git: 39e4665c9694 - main - PP mutexes: lock: Reduce = 'umtx_lock' holding before taking the user lock Olivier Certner >=20 > These changes are not in 14.0-RELEASE but are in 14.1-STABLE and = 14.1-RELEASE. > So I expect that when any ampere*'s progress to 14.1-RELEASE the armv7 > problems would start for them. >=20 > These changes are not in 13.3-RELEASE but are in 13.3-STABLE. So I = expect that > when any ampere*'s progress to 13.4-RELEASE the problems would start = for them. >=20 >=20 > With the prior packages already built in a prior poudriere-devel > run it turns out that just: >=20 > # poudriere bulk -j main-armv7-poud -i graphics/graphviz >=20 > replicates the problem: >=20 > . . . > [00:00:45] Installing graphics/graphviz | graphviz-9.0.0_4 > [aarch64PBase] Installing graphviz-9.0.0_4... > [aarch64PBase] `-- Installing cairo-1.17.4_2,3... > [aarch64PBase] | `-- Installing fontconfig-2.15.0_2,1... > [aarch64PBase] | | `-- Installing expat-2.6.2... > [aarch64PBase] | | `-- Extracting expat-2.6.2: 100% > [aarch64PBase] | | `-- Installing freetype2-2.13.2... > [aarch64PBase] | | `-- Installing brotli-1.1.0,1... > [aarch64PBase] | | `-- Extracting brotli-1.1.0,1: 100% > [aarch64PBase] | | `-- Installing png-1.6.43... > [aarch64PBase] | | `-- Extracting png-1.6.43: 100% > [aarch64PBase] | | `-- Extracting freetype2-2.13.2: 100% > [aarch64PBase] | `-- Extracting fontconfig-2.15.0_2,1: 100% > [aarch64PBase] | `-- Installing glib-2.80.4,2... > [aarch64PBase] | | `-- Installing libffi-3.4.6... > [aarch64PBase] | | `-- Extracting libffi-3.4.6: 100% > [aarch64PBase] | | `-- Installing libiconv-1.17_1... > [aarch64PBase] | | `-- Extracting libiconv-1.17_1: 100% > [aarch64PBase] | | `-- Installing pcre2-10.43... > [aarch64PBase] | | `-- Extracting pcre2-10.43: 100% > [aarch64PBase] | | `-- Installing py311-packaging-24.1... > [aarch64PBase] | | `-- Installing python311-3.11.9_1... > [aarch64PBase] | | | `-- Installing mpdecimal-4.0.0... > [aarch64PBase] | | | `-- Extracting mpdecimal-4.0.0: 100% > [aarch64PBase] | | | `-- Installing readline-8.2.10... > [aarch64PBase] | | | `-- Extracting readline-8.2.10: 100% > [aarch64PBase] | | `-- Extracting python311-3.11.9_1: 100% > [aarch64PBase] | | `-- Extracting py311-packaging-24.1: 100% > [aarch64PBase] | `-- Extracting glib-2.80.4,2: 100% > [aarch64PBase] | `-- Installing libglvnd-1.7.0... > [aarch64PBase] | `-- Extracting libglvnd-1.7.0: 100% > [aarch64PBase] | `-- Installing pixman-0.42.2... > [aarch64PBase] | `-- Extracting pixman-0.42.2: 100% > [aarch64PBase] `-- Extracting cairo-1.17.4_2,3: 100% > [aarch64PBase] `-- Installing harfbuzz-9.0.0... > [aarch64PBase] | `-- Installing graphite2-1.3.14... > [aarch64PBase] | `-- Extracting graphite2-1.3.14: 100% > [aarch64PBase] `-- Extracting harfbuzz-9.0.0: 100% > [aarch64PBase] `-- Installing jpeg-turbo-3.0.3... > [aarch64PBase] `-- Extracting jpeg-turbo-3.0.3: 100% > [aarch64PBase] `-- Installing libgd-2.3.3_13,1... > [aarch64PBase] | `-- Installing tiff-4.6.0... > [aarch64PBase] | | `-- Installing jbigkit-2.1_3... > [aarch64PBase] | | `-- Extracting jbigkit-2.1_3: 100% > [aarch64PBase] | | `-- Installing lerc-4.0.0... > [aarch64PBase] | | `-- Extracting lerc-4.0.0: 100% > [aarch64PBase] | | `-- Installing libdeflate-1.20... > [aarch64PBase] | | `-- Extracting libdeflate-1.20: 100% > [aarch64PBase] | | `-- Installing zstd-1.5.6... > [aarch64PBase] | | `-- Installing liblz4-1.9.4_1,1... > [aarch64PBase] | | `-- Extracting liblz4-1.9.4_1,1: 100% > [aarch64PBase] | | `-- Extracting zstd-1.5.6: 100% > [aarch64PBase] | `-- Extracting tiff-4.6.0: 100% > [aarch64PBase] | `-- Installing webp-1.4.0_1... > [aarch64PBase] | | `-- Installing giflib-5.2.2... > [aarch64PBase] | | `-- Extracting giflib-5.2.2: 100% > [aarch64PBase] | `-- Extracting webp-1.4.0_1: 100% > [aarch64PBase] `-- Extracting libgd-2.3.3_13,1: 100% > [aarch64PBase] `-- Installing libltdl-2.4.7... > [aarch64PBase] `-- Extracting libltdl-2.4.7: 100% > [aarch64PBase] `-- Installing pango-1.52.2_1... > [aarch64PBase] | `-- Installing fribidi-1.0.15... > [aarch64PBase] | `-- Extracting fribidi-1.0.15: 100% > [aarch64PBase] | `-- Installing libXft-2.3.8... > [aarch64PBase] | `-- Extracting libXft-2.3.8: 100% > [aarch64PBase] | `-- Installing libthai-0.1.29_1... > [aarch64PBase] | | `-- Installing libdatrie-0.2.13_2... > [aarch64PBase] | | `-- Extracting libdatrie-0.2.13_2: 100% > [aarch64PBase] | `-- Extracting libthai-0.1.29_1: 100% > [aarch64PBase] `-- Extracting pango-1.52.2_1: 100% > [aarch64PBase] Extracting graphviz-9.0.0_4: 100% >=20 > And here it is hung with /usr/local/bin/dot -c in urdlck : >=20 > 0 1483 4502 7 68 0 15760 4872 wait I+ 0 0:03.92 | | = `-- /usr/local/libexec/poudriere/sh -e -o pipefail = /usr/local/share/poudriere/bulk.sh -j main-armv7-poud -i = graphics/graphviz > 0 1894 1483 5 68 0 15760 4712 nanslp S 0 0:02.07 | | = |-- sh: poudriere[main-armv7-poud-default]: html_json_main (sh) > 0 25321 1483 6 68 0 6664 3868 wait I+J 0 0:00.11 | | = `-- /usr/bin/make -C /usr/ports/graphics/graphviz install-package > 0 25322 25321 5 68 0 11140 8860 wait I+J 0 0:00.00 | | = `-- /usr/local/sbin/pkg-static add /packages/All/graphviz-9.0.0_4.pkg > 0 25323 25322 5 20 0 63824 45144 select S+J 0 0:02.85 | | = `-- /usr/local/sbin/pkg-static add = /packages/All/graphviz-9.0.0_4.pkg > 0 26900 25323 3 68 0 26292 23804 urdlck I+J 0 0:00.02 | | = `-- /usr/local/bin/dot -c >=20 >=20 >> FYI for the replication that I got: >>=20 >> /usr/local/sbin/pkg-static add -A /packages/All/graphviz-9.0.0_4.pkg >> runs: >> /usr/local/bin/dot -c >>=20 >> each such /usr/local/bin/dot is stuck at MWCHAN urdlck . >>=20 >> # poudriere status -b >> [main-armv7-poud-default] [2024-07-16_04h27m31s] [parallel_build] = Queued: 449 Built: 433 Failed: 0 Skipped: 0 Ignored: 0 Fetched: 0 = Tobuild: 16 Time: 04:44:38 >> ID TOTAL ORIGIN PKGNAME = PHASE PHASE TMPFS CPU% MEM% >> [01] 00:59:45 graphics/rubygem-ruby-graphviz | = rubygem-ruby-graphviz-1.2.5 run-depends 00:59:30 1.59 GiB 0% 0.2% >> [02] 00:49:59 graphics/p5-GraphViz | p5-GraphViz-2.25 = build-depends 00:49:53 1.45 GiB 0% 0.2% >> [03] 00:59:45 graphics/py-pydot@py311 | py311-pydot-2.0.0 = run-depends 00:59:34 1.47 GiB 0% 0.2% >> [04] 00:59:45 graphics/py-pygraphviz@py311 | py311-pygraphviz-1.6 = lib-depends 00:59:33 1.47 GiB 0% 0.2% >> [05] 00:58:57 graphics/py-graphviz@py311 | py311-graphviz-0.10.1 = run-depends 00:58:49 1.47 GiB 0% 0.2% >> [06] 00:59:20 audio/ganv | ganv-1.8.2_1 = lib-depends 00:59:10 1.53 GiB 0% 0.2% >> [07] 00:59:45 devel/libr3 | libr3-1.0.0_2 = lib-depends 00:59:19 1.54 GiB 0% 0.3% >> [08] 00:59:45 net/netmap | netmap-0.1.3_2 = run-depends 00:59:22 1.46 GiB 0% 0.3% >>=20 >> I had started the bulk build via the list: >>=20 >> # more ~/origins/ampere2-failures-armv7.txt >> audio/ganv >> devel/doxygen >> devel/libr3 >> graphics/p5-GraphViz >> graphics/p5-GraphViz2 >> graphics/oyranos >> graphics/pear-Image_GraphViz@php81 >> graphics/py-graphviz@py311 >> graphics/py-pydot@py311 >> graphics/py-pygraphviz@py311 >> graphics/rubygem-ruby-graphviz >> math/ggobi >> net-mgmt/librenms >> net/netmap >> print/dot2tex@py311 >>=20 >> # poudriere bulk -j main-armv7-poud `cat = ~/origins/ampere2-failures-armv7.txt` >> . . . >> [00:00:12] Building 449 packages using up to 8 builders >> . . . >> [03:44:55] [01] [00:18:54] Finished graphics/graphviz | = graphviz-9.0.0_4: Success >> [03:44:56] [01] [00:00:00] Building graphics/rubygem-ruby-graphviz = | rubygem-ruby-graphviz-1.2.5 >> [03:44:56] [03] [00:00:00] Building graphics/py-pydot@py311 | = py311-pydot-2.0.0 >> [03:44:56] [04] [00:00:00] Building graphics/py-pygraphviz@py311 | = py311-pygraphviz-1.6 >> [03:44:56] [07] [00:00:00] Building devel/libr3 | libr3-1.0.0_2 >> [03:44:56] [08] [00:00:00] Building net/netmap | netmap-0.1.3_2 >> [03:45:21] [06] [00:09:33] Finished x11-toolkits/gtkmm24 | = gtkmm24-2.24.5_4: Success >> [03:45:21] [06] [00:00:00] Building audio/ganv | ganv-1.8.2_1 >> [03:45:44] [05] [00:13:33] Finished graphics/ImageMagick6@nox11 | = ImageMagick6-nox11-6.9.12.77_9,1: Success >> [03:45:44] [05] [00:00:00] Building graphics/py-graphviz@py311 | = py311-graphviz-0.10.1 >> [03:54:42] [02] [00:24:53] Finished print/texlive-base | = texlive-base-20240312: Success >> [03:54:42] [02] [00:00:00] Building graphics/p5-GraphViz | = p5-GraphViz-2.25 >>=20 >> In /usr/src/sys/kern/kern_umtx.c there is: >>=20 >> static int >> do_rw_rdlock(struct thread *td, struct urwlock *rwlock, long fflag, >> struct _umtx_time *timeout) >> { >> . . . >> /* >> * Contention bit is set, before sleeping, increase >> * read waiter count. >> */ >> rv =3D fueword32(&rwlock->rw_blocked_readers, >> &blocked_readers); >> if (rv =3D=3D 0) >> rv =3D suword32(&rwlock->rw_blocked_readers, >> blocked_readers + 1); >> if (rv =3D=3D -1) { >> umtxq_unbusy_unlocked(&uq->uq_key); >> error =3D EFAULT; >> break; >> } >> while (state & wrflags) { >> umtxq_lock(&uq->uq_key); >> umtxq_insert(uq); >> umtxq_unbusy(&uq->uq_key); >> error =3D umtxq_sleep(uq, "urdlck", timeout = =3D=3D NULL ? >> NULL : &timo); >> umtxq_busy(&uq->uq_key); >> umtxq_remove(uq); >> umtxq_unlock(&uq->uq_key); >> if (error) >> break; >> rv =3D fueword32(&rwlock->rw_state, &state); >> if (rv =3D=3D -1) { >> error =3D EFAULT; >> break; >> } >> } >>=20 >> . . . >>=20 >>=20 >>=20 >> For reference: >>=20 >> # ps -alxdww | less >> UID PID PPID C PRI NI VSZ RSS MWCHAN STAT TT TIME = COMMAND >> . . . >> 0 87700 4522 6 20 0 16576 1888 - T 0 0:00.01 | | = |-- vi /usr/local/share/poudriere/jail.sh >> 0 91496 4522 4 20 0 15760 4684 select S+ 0 0:06.88 | | = `-- /usr/local/libexec/poudriere/sh -e -o pipefail = /usr/local/share/poudriere/bulk.sh audio/ganv devel/doxygen devel/libr3 = graphics/p5-GraphViz graphics/p5-GraphViz2 graphics/oyranos = graphics/pear-Image_GraphViz@php81 graphics/py-graphviz@py311 = graphics/py-pydot@py311 graphics/py-pygraphviz@py311 = graphics/rubygem-ruby-graphviz math/ggobi net-mgmt/librenms net/netmap = print/dot2tex@py311 >> 0 37688 91496 0 68 0 15760 4700 wait I 0 0:00.05 | | = |-- sh: poudriere[main-armv7-poud-default][01]: build_pkg = (rubygem-ruby-graphviz-1.2.5) (sh) >> 0 47568 37688 0 68 0 6664 3664 wait IJ 0 0:00.03 | | = | `-- /usr/bin/make -C /usr/ports/graphics/rubygem-ruby-graphviz = run-depends >> 0 47598 47568 6 68 0 5568 2988 wait IJ 0 0:00.01 | | = | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh >> 0 47743 47598 6 68 0 11188 8864 wait IJ 0 0:00.00 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >> 0 47747 47743 3 20 0 71692 48984 select SJ 0 0:04.26 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >> 0 56383 47747 1 68 0 26292 23812 urdlck IJ 0 0:00.02 | | = | `-- /usr/local/bin/dot -c >> 0 37700 91496 6 68 0 15760 4700 wait I 0 0:00.04 | | = |-- sh: poudriere[main-armv7-poud-default][03]: build_pkg = (py311-pydot-2.0.0) (sh) >> 0 45102 37700 2 68 0 6668 3704 wait IJ 0 0:00.02 | | = | `-- /usr/bin/make -C /usr/ports/graphics/py-pydot FLAVOR=3Dpy311 = run-depends >> 0 45156 45102 4 68 0 5584 2992 wait IJ 0 0:00.01 | | = | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh >> 0 45215 45156 4 68 0 11144 8864 wait IJ 0 0:00.00 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >> 0 45218 45215 4 20 0 51420 31512 select SJ 0 0:02.68 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >> 0 52147 45218 2 68 0 26292 23812 urdlck IJ 0 0:00.02 | | = | `-- /usr/local/bin/dot -c >> 0 37721 91496 1 68 0 15760 4700 wait I 0 0:00.04 | | = |-- sh: poudriere[main-armv7-poud-default][04]: build_pkg = (py311-pygraphviz-1.6) (sh) >> 0 45937 37721 1 68 0 6684 3744 wait IJ 0 0:00.03 | | = | `-- /usr/bin/make -C /usr/ports/graphics/py-pygraphviz FLAVOR=3Dpy311= lib-depends >> 0 46009 45937 7 68 0 5584 2992 wait IJ 0 0:00.01 | | = | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh >> 0 46127 46009 7 68 0 11144 8864 wait IJ 0 0:00.00 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >> 0 46129 46127 1 20 0 51384 31548 select SJ 0 0:02.73 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >> 0 53311 46129 4 68 0 26292 23812 urdlck IJ 0 0:00.02 | | = | `-- /usr/local/bin/dot -c >> 0 37744 91496 7 45 0 15760 4692 wait I 0 0:00.04 | | = |-- sh: poudriere[main-armv7-poud-default][07]: build_pkg = (libr3-1.0.0_2) (sh) >> 0 55198 37744 0 50 0 6664 3664 wait IJ 0 0:00.03 | | = | `-- /usr/bin/make -C /usr/ports/devel/libr3 lib-depends >> 0 55229 55198 0 68 0 5588 2988 wait IJ 0 0:00.01 | | = | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh >> 0 55594 55229 7 68 0 11168 8864 wait IJ 0 0:00.00 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >> 0 55596 55594 2 20 0 69796 50180 select SJ 0 0:04.53 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >> 0 62753 55596 0 68 0 26292 23812 urdlck IJ 0 0:00.02 | | = | `-- /usr/local/bin/dot -c >> 0 37763 91496 4 29 0 15760 4696 wait I 0 0:00.05 | | = |-- sh: poudriere[main-armv7-poud-default][08]: build_pkg = (netmap-0.1.3_2) (sh) >> 0 51054 37763 6 36 0 6636 3684 wait IJ 0 0:00.03 | | = | `-- /usr/bin/make -C /usr/ports/net/netmap run-depends >> 0 51107 51054 3 68 0 5568 2988 wait IJ 0 0:00.01 | | = | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh >> 0 51576 51107 3 68 0 11168 8860 wait IJ 0 0:00.00 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >> 0 51580 51576 3 20 0 68220 49432 select SJ 0 0:04.27 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >> 0 59063 51580 6 68 0 26292 23808 urdlck IJ 0 0:00.02 | | = | `-- /usr/local/bin/dot -c >> 0 53709 91496 1 68 0 15760 4700 wait I 0 0:00.04 | | = |-- sh: poudriere[main-armv7-poud-default][06]: build_pkg = (ganv-1.8.2_1) (sh) >> 0 63371 53709 6 68 0 6636 3668 wait IJ 0 0:00.03 | | = | `-- /usr/bin/make -C /usr/ports/audio/ganv lib-depends >> 0 63377 63371 1 68 0 5580 2996 wait IJ 0 0:00.01 | | = | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh >> 0 63413 63377 4 68 0 11180 8864 wait IJ 0 0:00.00 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >> 0 63414 63413 3 20 0 56212 35980 select SJ 0 0:02.21 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >> 0 65577 63414 6 68 0 26292 23812 urdlck IJ 0 0:00.01 | | = | `-- /usr/local/bin/dot -c >> 0 63365 91496 3 68 0 15760 4696 wait I 0 0:00.03 | | = |-- sh: poudriere[main-armv7-poud-default][02]: build_pkg = (p5-GraphViz-2.25) (sh) >> 0 63807 63365 7 68 0 6696 3672 wait IJ 0 0:00.02 | | = | `-- /usr/bin/make -C /usr/ports/graphics/p5-GraphViz build-depends >> 0 63808 63807 2 68 0 5568 2988 wait IJ 0 0:00.01 | | = | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh >> 0 63833 63808 4 68 0 11188 8864 wait IJ 0 0:00.00 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >> 0 63834 63833 7 20 0 67400 48532 select SJ 0 0:03.52 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >> 0 66041 63834 2 68 0 26292 23812 urdlck IJ 0 0:00.02 | | = | `-- /usr/local/bin/dot -c >> 0 69974 91496 1 68 0 15760 4700 wait I 0 0:00.04 | | = |-- sh: poudriere[main-armv7-poud-default][05]: build_pkg = (py311-graphviz-0.10.1) (sh) >> 0 73474 69974 5 68 0 6684 3740 wait IJ 0 0:00.02 | | = | `-- /usr/bin/make -C /usr/ports/graphics/py-graphviz FLAVOR=3Dpy311 = run-depends >> 0 73496 73474 6 68 0 5584 2992 wait IJ 0 0:00.01 | | = | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh >> 0 73521 73496 7 68 0 11144 8864 wait IJ 0 0:00.00 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >> 0 73522 73521 0 20 0 52432 32664 select SJ 0 0:02.70 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >> 0 76540 73522 3 68 0 26292 23812 urdlck IJ 0 0:00.02 | | = | `-- /usr/local/bin/dot -c >> 0 91907 91496 5 68 0 15760 4492 nanslp S 0 1:05.17 | | = |-- sh: poudriere[main-armv7-poud-default]: html_json_main (sh) >> 0 99134 91496 1 40 0 15760 4740 piperd I 0 0:03.22 | | = `-- sh: poudriere[main-armv7-poud-default]: pkg_cacher_main (sh) >> 0 23 A little bit more context for /usr/local/bin/dot : 0x20631520 in _umtx_op () from /lib/libsys.so.7 (gdb) bt #0 0x20631520 in _umtx_op () at /lib/libsys.so.7 #1 0x2063245c in _umtx_op_err () at /lib/libsys.so.7 #2 0x203a2da8 in ??? () at /lib/libthr.so.3 #3 0x2039bbf4 in ??? () at /lib/libthr.so.3 #4 0x20061788 in ??? () at /libexec/ld-elf.so.1 And the associated instance of /usr/local/sbin/pkg-static : (gdb) bt #0 _poll () at _poll.S:4 #1 0x007669e0 in __thr_poll (fds=3D0xd1, nfds=3D1, timeout=3D1000) at = /home/pkgbuild/worktrees/main/lib/libthr/thread/thr_syscalls.c:320 #2 0x003602e8 in pkg_script_run_child (pid=3D64019, = pstat=3Dpstat@entry=3D0xffffc49c, inputfd=3D9, script_name=3D0x5d5c9 = "POST-INSTALL") at scripts.c:303 #3 0x0035fc34 in pkg_script_run (pkg=3D0x20972e00, type=3D, upgrade=3D) at scripts.c:227 #4 0x00371250 in pkg_add_common (db=3D, path=3D, path@entry=3D0xffffda7f "/packages/All/graphviz-9.0.0_4.pkg", = flags=3D, reloc=3D, remote=3D0x0, = local=3D0x0, t=3D0x0) at pkg_add.c:1386 #5 0x003707e4 in pkg_add (db=3D0x4, path=3D0x1 , path@entry=3D0xffffda7f = "/packages/All/graphviz-9.0.0_4.pkg", flags=3D1000, = location=3D0x766990 <__thr_poll> "\360H-\351\020\260\215\342") at = pkg_add.c:1460 #6 0x00194544 in exec_add (argc=3D, argv=3D) at add.c:178 #7 0x0019f840 in main (argc=3D2, argv=3D0xffffd87c) at main.c:872 =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Wed Jul 17 06:45:04 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WP5zQ548Kz5QgHZ for ; Wed, 17 Jul 2024 06:45:22 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic303-23.consmr.mail.gq1.yahoo.com (sonic303-23.consmr.mail.gq1.yahoo.com [98.137.64.204]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4WP5zP4Q72z47ZP for ; Wed, 17 Jul 2024 06:45:21 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=cRvlMcOO; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.64.204 as permitted sender) smtp.mailfrom=marklmi@yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1721198719; bh=9Puj9o0GWAC2Iq8L3/sJ4Fi8jp2U8o0weg+wrpzPFOA=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=cRvlMcOOCFug6KHvKJ/5Lf3Sw69T36B8kuZ6m3Nv7zYGPCfYLcnuN21y3ubFCll7uvhMgyak9RhvIFHELhT+z6z3DP/vIyTUr+SFqe5pfVaQYBkftcYvbno3WfRwE90HvKukaVFezn8laBkb33uU0ZhHoQUByimfC0e6HO5db91hSHoI30V7E8CWAyFKZ+HX+vdhrCvWjcrF5PVrJ1tNbseZvqxPxogqgHwsF6PSZk9G0LgGj9+IkrOUQkzh1XwMViK2ifelDs2qmz4kAjBZT+yBpLQ+F0C0iYWdIj8WCCySZmUf+/8xFXjy4rKcrK+h2kb5fKNH+8foOnZ1w7zC1Q== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1721198719; bh=IiP9y+sBqjHwbrpc9m9Gv1jY0Z9mKNvCgIMgIgBZNAT=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=Iu9L1e6pJD3bqzKRaDGn/lOqTanh6JlzTKQTS4HULAFBsRbzxpbCpmCRJ5qSIcmzOtpiOPmQYLwy7vDSP3DWx8ljvOTn2V3NTN1F7YQE6r0ci1/f3p5w6bGusXpvPhXdvj7rb1J5tPqwONdxWlkuc0nnBVKULpyrMfVLTyo46fUYMYBMMgN2AOPuCXWn5WNPAz6HT5tkDXeJkWxweCN7QFWmwDsAMIOugA8Q9sWx8CH5oQboAYEex87gDPvCuh1IQbRl635XH0/HLaGTnHH758cvBUlMGO9HlS68s13pPNtSPbSLFPPbf+j0J674rI9HvUcpRPVWBrTnIV/dFsRqvw== X-YMail-OSG: acpanFwVM1luJq615p_96UTOooE958nhE0yCA_9wJoMZ2eFqXYMh1uyp84TrOwL Npz3JE4bpUPpWscBBKKW0C6m4OsTM4at13_7Gd33hQLj5HJit5kS8AgeO0I16kj6ta6UfCXjCI7c NoJMrgruXRQLwAIQ8FpkfaFkrmxNLkW9QVFBqScpiNHj0q2E3z32BfAIy_lIvKS3dYGbYFDfuJEF Q7S1xRXgCKEKADpwnCAcyLjKiAJXbWdwKHFlU6dTP.naRAG_2MCB12krGupyFo7BTWEqoP2hlSMQ fmBcWF0SSiaPfSHMOJMUFunhDQDwqW.3.OJgPWHSIwHeUnZ77.vu6JJrdLLAyIHm10W6YLRI2ooV c6FMc3xB12_2x3.DH_fR5gUIeOkLAwlgsvVUVkVF4aTGcr3j9wVWMqlXVNyRcHff.MC7o_YC7s65 Z6saR8FbQoyDUoVSHKa0MJTJX6hjpTzMN79OPUjL_C6WV6OOdp6r.2o.nJJYAh3Q.WXevH_O7_qn RfOJQE5pYFv.w2rqRuJ549gqKm2tbju5geBvS2CosRPanQaNkuEamn2Vb8YMXq4fcg70Qj52l.1J pwLOZpnJEsfqWeMxc6J4SOHW6vD0odrGQ0DNAPMdc1kztEU5G8inVShcwg4ssVEXxSxSsQen_uf0 2fzaNjlFOkVzxIB55DSF5GDujGvJ.INiKEIhEgwHEm9cGN8PErBNhE0TPYGcmdvDjolJf33bt4tN hQlBAX.IHcqp7NRwsp3OuIEUirFAzXMSIqgsb4I448new3k1LB9HPSvyQFMgXvVT2g6G6mKYKATu cbrXapY9Vj4iNV5AbxxH4ACcHkgJJ1Cw6Lvt9uSLBFvvdrm9FL_dt.I00p7XmyK8Q6swKLWc7g62 FrSgqIXMY4onlToVIZLTwBqgoyAJGSbUnQwqa4vHGPJuDkYZcWPqt9hy0yS_2FH5_986R.32Ck73 6qZMFjDQzskjeqjJb1sPmA4tgbT80ybh8KRPFdaqqg3IAow4ldctI5VS8TZMEIGTQFKxQqLGvA84 NiGbRE1vvHvAzMXxvbgBzBexXG7kp5dxnhk.iLz.6C.opiPn5HCWoj5ynhIkX16ZVSlQ9wOOAQdN 2niYExJq33f5FZvbr6tvI7Nfz6D2CT6WR1YyOjHXzJPYo0HslckMAp6y2aWNsE15nVQ.9wpg8v2o 57SgHpG2tMvTK.3v.pnyUg4c6Af7I_RKV3ayvlv2O2E2P790M28C2HO404LUzcFXVnHpKzrNsmyW FW4vq.psEBoxgP5D2UEIsb4_pdculN8bmvf5v38BZjnRBtHnwNbsnVrX1Fo9dz87uhByNktS5klV cjlMWLyzpIVClBncEtN_Srl3FiGfyG_cfnp4VQtkoEL2espSACkfK_iBrKiJfCQn.DS_qloG8kad 91SHPCwNqv8ECfm1LranECtRxIC7DFM4Xasdhv8L3IkxyPu.l5x1vcBBbQgHq6zqPOWTmwxlZCCt gOLW4UU7S0XfHCqQbkbXSBAWB6B8VlBlC9ezTcGZh4M_BHnqxKuN9jtgcW6VNrH4cLx4pHRMEfbZ qfDyJ6q.3GvQoFwFURfWcyszYgU6M0oGe1KlULLl4ubzo4axIa4V0FAXwTmWUOX5DhFGSA1FM6UV 2uLdRPbTWSqhfLKyN0ZeOkjzy4op1NI0MXEJyLcajcZYuga7utQNHvXtYnlH9N5QnyhJwXWoEep7 QOOx875F9YRLVpjx17htiXwRE46Xl6YNTNAl2QO2gOOSbkoqeJW5hHHP9a.cdK.Pdh7SvD.QRcGJ zOHHAfECIshnvAlOxRH_3YYTNk1e_tOjmRDjslv3btqD5_VBE_RkieRGMifDrVgGOWDgH1NJ2ph8 zCLry9KOZCVPO4QUtmggn5a4A523xjkbn.2v_p43sylpFhFEHczBPc2qSdiJNc.e5NdRD8R7x81T 0nb4.NTI9XIWCzLp2Gvqgx5LGPfowUwwu_THtjASoPljQWm2O4WedQ9wHvH2OGA66y13Gpq4dmCb EOY3tAUPVpcjk8F.qIZ1P7xoJ1OnZghr0yI87wEZobrp3C0bjLLsCxWgCjOGIru2kqiM6c43lbLG 7vGmZKunfPUVj5uyh6LaAAx.jJlUbZaC.T3KaNoXN1SKdFpISTohZS.8LhOFIcqTWkU_uBEDFOUB 1n.nemIavtskKDjH4Kijtrb0FrA0MkJvyxEJnt64iMihankmRmgU.3HUT36DFn5g3fkgiKgj76aB D8qSX4S7HRtydoTyd.lXYgXD.kYYgYfD6m8ZF5hrFHV_refyI7PmY0fYQihVwDgvtZ.3xUvCnkL8 4k1SrWGMQY6gT X-Sonic-MF: X-Sonic-ID: 49031e85-ab59-43d1-b41d-7a5fc8ce7d0c Received: from sonic.gate.mail.ne1.yahoo.com by sonic303.consmr.mail.gq1.yahoo.com with HTTP; Wed, 17 Jul 2024 06:45:19 +0000 Received: by hermes--production-gq1-799bb7c8cf-jmg5m (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID b047c9622ebb6340b079d36cac396459; Wed, 17 Jul 2024 06:45:15 +0000 (UTC) Content-Type: text/plain; charset=utf-8 List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.600.62\)) Subject: Re: armv7-on-aarch64 stuck at urdlck: I got a replication of the "ampere2" bulk build hangup problem on a Windows DevKit 2023 From: Mark Millard In-Reply-To: Date: Tue, 16 Jul 2024 23:45:04 -0700 Cc: Konstantin Belousov , Baptiste Daroussin Content-Transfer-Encoding: quoted-printable Message-Id: References: <8214703E-AB28-4FB3-A3DD-03C87363D8C6@yahoo.com> To: "olce@freebsd.org" , Philip Paeps , FreeBSD ARM List , FreeBSD Mailing List , Current FreeBSD X-Mailer: Apple Mail (2.3774.600.62) X-Spamd-Bar: --- X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; DKIM_TRACE(0.00)[yahoo.com:+]; FREEMAIL_FROM(0.00)[yahoo.com]; TO_DN_SOME(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.204:from]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MLMMJ_DEST(0.00)[freebsd-arm@freebsd.org]; RCPT_COUNT_SEVEN(0.00)[7]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.64.204:from]; APPLE_MAILER_COMMON(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim] X-Rspamd-Queue-Id: 4WP5zP4Q72z47ZP On Jul 16, 2024, at 18:41, Mark Millard wrote: > On Jul 16, 2024, at 11:37, Mark Millard wrote: >=20 >> On Jul 16, 2024, at 10:42, Mark Millard wrote: >>=20 >>> No longer is the problem only observed on ampere2! But this was with >>> a non-debug, personally built kernel that has some of my now = patches. >>> I'll see if I can replicate the issue with an official pkgbase debug >>> kernel. >>=20 >> It replicated with the official pkgbase debug kernel. The >> kernel did not report anything. >>=20 >> The following commits in main and happen between the last working >> ampere2 armv7 builds and the first failing ampere2 builds and >> look be the only likely contributors from that range as far as >> I could tell: >>=20 >> Tue, 27 Feb 2024 >> . . . >> =E2=80=A2 git: 1df8700aa6cf - main - PP mutexes: unlock: Reset = inherited prio regardless of privileges Olivier Certner=20 >> =E2=80=A2 git: 9ac3ac9ece62 - main - PP mutexes: lock: Check if = priority is too high against base one Olivier Certner=20 >> =E2=80=A2 git: 39e4665c9694 - main - PP mutexes: lock: Reduce = 'umtx_lock' holding before taking the user lock Olivier Certner >>=20 >> These changes are not in 14.0-RELEASE but are in 14.1-STABLE and = 14.1-RELEASE. >> So I expect that when any ampere*'s progress to 14.1-RELEASE the = armv7 >> problems would start for them. >>=20 >> These changes are not in 13.3-RELEASE but are in 13.3-STABLE. So I = expect that >> when any ampere*'s progress to 13.4-RELEASE the problems would start = for them. >>=20 >>=20 >> With the prior packages already built in a prior poudriere-devel >> run it turns out that just: >>=20 >> # poudriere bulk -j main-armv7-poud -i graphics/graphviz >>=20 >> replicates the problem: >>=20 >> . . . >> [00:00:45] Installing graphics/graphviz | graphviz-9.0.0_4 >> [aarch64PBase] Installing graphviz-9.0.0_4... >> [aarch64PBase] `-- Installing cairo-1.17.4_2,3... >> [aarch64PBase] | `-- Installing fontconfig-2.15.0_2,1... >> [aarch64PBase] | | `-- Installing expat-2.6.2... >> [aarch64PBase] | | `-- Extracting expat-2.6.2: 100% >> [aarch64PBase] | | `-- Installing freetype2-2.13.2... >> [aarch64PBase] | | `-- Installing brotli-1.1.0,1... >> [aarch64PBase] | | `-- Extracting brotli-1.1.0,1: 100% >> [aarch64PBase] | | `-- Installing png-1.6.43... >> [aarch64PBase] | | `-- Extracting png-1.6.43: 100% >> [aarch64PBase] | | `-- Extracting freetype2-2.13.2: 100% >> [aarch64PBase] | `-- Extracting fontconfig-2.15.0_2,1: 100% >> [aarch64PBase] | `-- Installing glib-2.80.4,2... >> [aarch64PBase] | | `-- Installing libffi-3.4.6... >> [aarch64PBase] | | `-- Extracting libffi-3.4.6: 100% >> [aarch64PBase] | | `-- Installing libiconv-1.17_1... >> [aarch64PBase] | | `-- Extracting libiconv-1.17_1: 100% >> [aarch64PBase] | | `-- Installing pcre2-10.43... >> [aarch64PBase] | | `-- Extracting pcre2-10.43: 100% >> [aarch64PBase] | | `-- Installing py311-packaging-24.1... >> [aarch64PBase] | | `-- Installing python311-3.11.9_1... >> [aarch64PBase] | | | `-- Installing mpdecimal-4.0.0... >> [aarch64PBase] | | | `-- Extracting mpdecimal-4.0.0: 100% >> [aarch64PBase] | | | `-- Installing readline-8.2.10... >> [aarch64PBase] | | | `-- Extracting readline-8.2.10: 100% >> [aarch64PBase] | | `-- Extracting python311-3.11.9_1: 100% >> [aarch64PBase] | | `-- Extracting py311-packaging-24.1: 100% >> [aarch64PBase] | `-- Extracting glib-2.80.4,2: 100% >> [aarch64PBase] | `-- Installing libglvnd-1.7.0... >> [aarch64PBase] | `-- Extracting libglvnd-1.7.0: 100% >> [aarch64PBase] | `-- Installing pixman-0.42.2... >> [aarch64PBase] | `-- Extracting pixman-0.42.2: 100% >> [aarch64PBase] `-- Extracting cairo-1.17.4_2,3: 100% >> [aarch64PBase] `-- Installing harfbuzz-9.0.0... >> [aarch64PBase] | `-- Installing graphite2-1.3.14... >> [aarch64PBase] | `-- Extracting graphite2-1.3.14: 100% >> [aarch64PBase] `-- Extracting harfbuzz-9.0.0: 100% >> [aarch64PBase] `-- Installing jpeg-turbo-3.0.3... >> [aarch64PBase] `-- Extracting jpeg-turbo-3.0.3: 100% >> [aarch64PBase] `-- Installing libgd-2.3.3_13,1... >> [aarch64PBase] | `-- Installing tiff-4.6.0... >> [aarch64PBase] | | `-- Installing jbigkit-2.1_3... >> [aarch64PBase] | | `-- Extracting jbigkit-2.1_3: 100% >> [aarch64PBase] | | `-- Installing lerc-4.0.0... >> [aarch64PBase] | | `-- Extracting lerc-4.0.0: 100% >> [aarch64PBase] | | `-- Installing libdeflate-1.20... >> [aarch64PBase] | | `-- Extracting libdeflate-1.20: 100% >> [aarch64PBase] | | `-- Installing zstd-1.5.6... >> [aarch64PBase] | | `-- Installing liblz4-1.9.4_1,1... >> [aarch64PBase] | | `-- Extracting liblz4-1.9.4_1,1: 100% >> [aarch64PBase] | | `-- Extracting zstd-1.5.6: 100% >> [aarch64PBase] | `-- Extracting tiff-4.6.0: 100% >> [aarch64PBase] | `-- Installing webp-1.4.0_1... >> [aarch64PBase] | | `-- Installing giflib-5.2.2... >> [aarch64PBase] | | `-- Extracting giflib-5.2.2: 100% >> [aarch64PBase] | `-- Extracting webp-1.4.0_1: 100% >> [aarch64PBase] `-- Extracting libgd-2.3.3_13,1: 100% >> [aarch64PBase] `-- Installing libltdl-2.4.7... >> [aarch64PBase] `-- Extracting libltdl-2.4.7: 100% >> [aarch64PBase] `-- Installing pango-1.52.2_1... >> [aarch64PBase] | `-- Installing fribidi-1.0.15... >> [aarch64PBase] | `-- Extracting fribidi-1.0.15: 100% >> [aarch64PBase] | `-- Installing libXft-2.3.8... >> [aarch64PBase] | `-- Extracting libXft-2.3.8: 100% >> [aarch64PBase] | `-- Installing libthai-0.1.29_1... >> [aarch64PBase] | | `-- Installing libdatrie-0.2.13_2... >> [aarch64PBase] | | `-- Extracting libdatrie-0.2.13_2: 100% >> [aarch64PBase] | `-- Extracting libthai-0.1.29_1: 100% >> [aarch64PBase] `-- Extracting pango-1.52.2_1: 100% >> [aarch64PBase] Extracting graphviz-9.0.0_4: 100% >>=20 >> And here it is hung with /usr/local/bin/dot -c in urdlck : >>=20 >> 0 1483 4502 7 68 0 15760 4872 wait I+ 0 0:03.92 | | = `-- /usr/local/libexec/poudriere/sh -e -o pipefail = /usr/local/share/poudriere/bulk.sh -j main-armv7-poud -i = graphics/graphviz >> 0 1894 1483 5 68 0 15760 4712 nanslp S 0 0:02.07 | | = |-- sh: poudriere[main-armv7-poud-default]: html_json_main (sh) >> 0 25321 1483 6 68 0 6664 3868 wait I+J 0 0:00.11 | | = `-- /usr/bin/make -C /usr/ports/graphics/graphviz install-package >> 0 25322 25321 5 68 0 11140 8860 wait I+J 0 0:00.00 | | = `-- /usr/local/sbin/pkg-static add /packages/All/graphviz-9.0.0_4.pkg >> 0 25323 25322 5 20 0 63824 45144 select S+J 0 0:02.85 | | = `-- /usr/local/sbin/pkg-static add = /packages/All/graphviz-9.0.0_4.pkg >> 0 26900 25323 3 68 0 26292 23804 urdlck I+J 0 0:00.02 | | = `-- /usr/local/bin/dot -c >>=20 >>=20 >>> FYI for the replication that I got: >>>=20 >>> /usr/local/sbin/pkg-static add -A /packages/All/graphviz-9.0.0_4.pkg >>> runs: >>> /usr/local/bin/dot -c >>>=20 >>> each such /usr/local/bin/dot is stuck at MWCHAN urdlck . >>>=20 >>> # poudriere status -b >>> [main-armv7-poud-default] [2024-07-16_04h27m31s] [parallel_build] = Queued: 449 Built: 433 Failed: 0 Skipped: 0 Ignored: 0 Fetched: 0 = Tobuild: 16 Time: 04:44:38 >>> ID TOTAL ORIGIN PKGNAME = PHASE PHASE TMPFS CPU% MEM% >>> [01] 00:59:45 graphics/rubygem-ruby-graphviz | = rubygem-ruby-graphviz-1.2.5 run-depends 00:59:30 1.59 GiB 0% 0.2% >>> [02] 00:49:59 graphics/p5-GraphViz | p5-GraphViz-2.25 = build-depends 00:49:53 1.45 GiB 0% 0.2% >>> [03] 00:59:45 graphics/py-pydot@py311 | py311-pydot-2.0.0 = run-depends 00:59:34 1.47 GiB 0% 0.2% >>> [04] 00:59:45 graphics/py-pygraphviz@py311 | py311-pygraphviz-1.6 = lib-depends 00:59:33 1.47 GiB 0% 0.2% >>> [05] 00:58:57 graphics/py-graphviz@py311 | py311-graphviz-0.10.1 = run-depends 00:58:49 1.47 GiB 0% 0.2% >>> [06] 00:59:20 audio/ganv | ganv-1.8.2_1 = lib-depends 00:59:10 1.53 GiB 0% 0.2% >>> [07] 00:59:45 devel/libr3 | libr3-1.0.0_2 = lib-depends 00:59:19 1.54 GiB 0% 0.3% >>> [08] 00:59:45 net/netmap | netmap-0.1.3_2 = run-depends 00:59:22 1.46 GiB 0% 0.3% >>>=20 >>> I had started the bulk build via the list: >>>=20 >>> # more ~/origins/ampere2-failures-armv7.txt >>> audio/ganv >>> devel/doxygen >>> devel/libr3 >>> graphics/p5-GraphViz >>> graphics/p5-GraphViz2 >>> graphics/oyranos >>> graphics/pear-Image_GraphViz@php81 >>> graphics/py-graphviz@py311 >>> graphics/py-pydot@py311 >>> graphics/py-pygraphviz@py311 >>> graphics/rubygem-ruby-graphviz >>> math/ggobi >>> net-mgmt/librenms >>> net/netmap >>> print/dot2tex@py311 >>>=20 >>> # poudriere bulk -j main-armv7-poud `cat = ~/origins/ampere2-failures-armv7.txt` >>> . . . >>> [00:00:12] Building 449 packages using up to 8 builders >>> . . . >>> [03:44:55] [01] [00:18:54] Finished graphics/graphviz | = graphviz-9.0.0_4: Success >>> [03:44:56] [01] [00:00:00] Building graphics/rubygem-ruby-graphviz = | rubygem-ruby-graphviz-1.2.5 >>> [03:44:56] [03] [00:00:00] Building graphics/py-pydot@py311 | = py311-pydot-2.0.0 >>> [03:44:56] [04] [00:00:00] Building graphics/py-pygraphviz@py311 | = py311-pygraphviz-1.6 >>> [03:44:56] [07] [00:00:00] Building devel/libr3 | libr3-1.0.0_2 >>> [03:44:56] [08] [00:00:00] Building net/netmap | netmap-0.1.3_2 >>> [03:45:21] [06] [00:09:33] Finished x11-toolkits/gtkmm24 | = gtkmm24-2.24.5_4: Success >>> [03:45:21] [06] [00:00:00] Building audio/ganv | ganv-1.8.2_1 >>> [03:45:44] [05] [00:13:33] Finished graphics/ImageMagick6@nox11 | = ImageMagick6-nox11-6.9.12.77_9,1: Success >>> [03:45:44] [05] [00:00:00] Building graphics/py-graphviz@py311 | = py311-graphviz-0.10.1 >>> [03:54:42] [02] [00:24:53] Finished print/texlive-base | = texlive-base-20240312: Success >>> [03:54:42] [02] [00:00:00] Building graphics/p5-GraphViz | = p5-GraphViz-2.25 >>>=20 >>> In /usr/src/sys/kern/kern_umtx.c there is: >>>=20 >>> static int >>> do_rw_rdlock(struct thread *td, struct urwlock *rwlock, long fflag, >>> struct _umtx_time *timeout) >>> { >>> . . . >>> /* >>> * Contention bit is set, before sleeping, increase >>> * read waiter count. >>> */ >>> rv =3D fueword32(&rwlock->rw_blocked_readers, >>> &blocked_readers); >>> if (rv =3D=3D 0) >>> rv =3D suword32(&rwlock->rw_blocked_readers, >>> blocked_readers + 1); >>> if (rv =3D=3D -1) { >>> umtxq_unbusy_unlocked(&uq->uq_key); >>> error =3D EFAULT; >>> break; >>> } >>> while (state & wrflags) { >>> umtxq_lock(&uq->uq_key); >>> umtxq_insert(uq); >>> umtxq_unbusy(&uq->uq_key); >>> error =3D umtxq_sleep(uq, "urdlck", timeout = =3D=3D NULL ? >>> NULL : &timo); >>> umtxq_busy(&uq->uq_key); >>> umtxq_remove(uq); >>> umtxq_unlock(&uq->uq_key); >>> if (error) >>> break; >>> rv =3D fueword32(&rwlock->rw_state, &state); >>> if (rv =3D=3D -1) { >>> error =3D EFAULT; >>> break; >>> } >>> } >>>=20 >>> . . . >>>=20 >>>=20 >>>=20 >>> For reference: >>>=20 >>> # ps -alxdww | less >>> UID PID PPID C PRI NI VSZ RSS MWCHAN STAT TT TIME = COMMAND >>> . . . >>> 0 87700 4522 6 20 0 16576 1888 - T 0 0:00.01 | | = |-- vi /usr/local/share/poudriere/jail.sh >>> 0 91496 4522 4 20 0 15760 4684 select S+ 0 0:06.88 | | = `-- /usr/local/libexec/poudriere/sh -e -o pipefail = /usr/local/share/poudriere/bulk.sh audio/ganv devel/doxygen devel/libr3 = graphics/p5-GraphViz graphics/p5-GraphViz2 graphics/oyranos = graphics/pear-Image_GraphViz@php81 graphics/py-graphviz@py311 = graphics/py-pydot@py311 graphics/py-pygraphviz@py311 = graphics/rubygem-ruby-graphviz math/ggobi net-mgmt/librenms net/netmap = print/dot2tex@py311 >>> 0 37688 91496 0 68 0 15760 4700 wait I 0 0:00.05 | | = |-- sh: poudriere[main-armv7-poud-default][01]: build_pkg = (rubygem-ruby-graphviz-1.2.5) (sh) >>> 0 47568 37688 0 68 0 6664 3664 wait IJ 0 0:00.03 | | = | `-- /usr/bin/make -C /usr/ports/graphics/rubygem-ruby-graphviz = run-depends >>> 0 47598 47568 6 68 0 5568 2988 wait IJ 0 0:00.01 | | = | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh >>> 0 47743 47598 6 68 0 11188 8864 wait IJ 0 0:00.00 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>> 0 47747 47743 3 20 0 71692 48984 select SJ 0 0:04.26 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>> 0 56383 47747 1 68 0 26292 23812 urdlck IJ 0 0:00.02 | | = | `-- /usr/local/bin/dot -c >>> 0 37700 91496 6 68 0 15760 4700 wait I 0 0:00.04 | | = |-- sh: poudriere[main-armv7-poud-default][03]: build_pkg = (py311-pydot-2.0.0) (sh) >>> 0 45102 37700 2 68 0 6668 3704 wait IJ 0 0:00.02 | | = | `-- /usr/bin/make -C /usr/ports/graphics/py-pydot FLAVOR=3Dpy311 = run-depends >>> 0 45156 45102 4 68 0 5584 2992 wait IJ 0 0:00.01 | | = | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh >>> 0 45215 45156 4 68 0 11144 8864 wait IJ 0 0:00.00 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>> 0 45218 45215 4 20 0 51420 31512 select SJ 0 0:02.68 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>> 0 52147 45218 2 68 0 26292 23812 urdlck IJ 0 0:00.02 | | = | `-- /usr/local/bin/dot -c >>> 0 37721 91496 1 68 0 15760 4700 wait I 0 0:00.04 | | = |-- sh: poudriere[main-armv7-poud-default][04]: build_pkg = (py311-pygraphviz-1.6) (sh) >>> 0 45937 37721 1 68 0 6684 3744 wait IJ 0 0:00.03 | | = | `-- /usr/bin/make -C /usr/ports/graphics/py-pygraphviz = FLAVOR=3Dpy311 lib-depends >>> 0 46009 45937 7 68 0 5584 2992 wait IJ 0 0:00.01 | | = | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh >>> 0 46127 46009 7 68 0 11144 8864 wait IJ 0 0:00.00 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>> 0 46129 46127 1 20 0 51384 31548 select SJ 0 0:02.73 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>> 0 53311 46129 4 68 0 26292 23812 urdlck IJ 0 0:00.02 | | = | `-- /usr/local/bin/dot -c >>> 0 37744 91496 7 45 0 15760 4692 wait I 0 0:00.04 | | = |-- sh: poudriere[main-armv7-poud-default][07]: build_pkg = (libr3-1.0.0_2) (sh) >>> 0 55198 37744 0 50 0 6664 3664 wait IJ 0 0:00.03 | | = | `-- /usr/bin/make -C /usr/ports/devel/libr3 lib-depends >>> 0 55229 55198 0 68 0 5588 2988 wait IJ 0 0:00.01 | | = | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh >>> 0 55594 55229 7 68 0 11168 8864 wait IJ 0 0:00.00 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>> 0 55596 55594 2 20 0 69796 50180 select SJ 0 0:04.53 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>> 0 62753 55596 0 68 0 26292 23812 urdlck IJ 0 0:00.02 | | = | `-- /usr/local/bin/dot -c >>> 0 37763 91496 4 29 0 15760 4696 wait I 0 0:00.05 | | = |-- sh: poudriere[main-armv7-poud-default][08]: build_pkg = (netmap-0.1.3_2) (sh) >>> 0 51054 37763 6 36 0 6636 3684 wait IJ 0 0:00.03 | | = | `-- /usr/bin/make -C /usr/ports/net/netmap run-depends >>> 0 51107 51054 3 68 0 5568 2988 wait IJ 0 0:00.01 | | = | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh >>> 0 51576 51107 3 68 0 11168 8860 wait IJ 0 0:00.00 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>> 0 51580 51576 3 20 0 68220 49432 select SJ 0 0:04.27 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>> 0 59063 51580 6 68 0 26292 23808 urdlck IJ 0 0:00.02 | | = | `-- /usr/local/bin/dot -c >>> 0 53709 91496 1 68 0 15760 4700 wait I 0 0:00.04 | | = |-- sh: poudriere[main-armv7-poud-default][06]: build_pkg = (ganv-1.8.2_1) (sh) >>> 0 63371 53709 6 68 0 6636 3668 wait IJ 0 0:00.03 | | = | `-- /usr/bin/make -C /usr/ports/audio/ganv lib-depends >>> 0 63377 63371 1 68 0 5580 2996 wait IJ 0 0:00.01 | | = | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh >>> 0 63413 63377 4 68 0 11180 8864 wait IJ 0 0:00.00 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>> 0 63414 63413 3 20 0 56212 35980 select SJ 0 0:02.21 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>> 0 65577 63414 6 68 0 26292 23812 urdlck IJ 0 0:00.01 | | = | `-- /usr/local/bin/dot -c >>> 0 63365 91496 3 68 0 15760 4696 wait I 0 0:00.03 | | = |-- sh: poudriere[main-armv7-poud-default][02]: build_pkg = (p5-GraphViz-2.25) (sh) >>> 0 63807 63365 7 68 0 6696 3672 wait IJ 0 0:00.02 | | = | `-- /usr/bin/make -C /usr/ports/graphics/p5-GraphViz build-depends >>> 0 63808 63807 2 68 0 5568 2988 wait IJ 0 0:00.01 | | = | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh >>> 0 63833 63808 4 68 0 11188 8864 wait IJ 0 0:00.00 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>> 0 63834 63833 7 20 0 67400 48532 select SJ 0 0:03.52 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>> 0 66041 63834 2 68 0 26292 23812 urdlck IJ 0 0:00.02 | | = | `-- /usr/local/bin/dot -c >>> 0 69974 91496 1 68 0 15760 4700 wait I 0 0:00.04 | | = |-- sh: poudriere[main-armv7-poud-default][05]: build_pkg = (py311-graphviz-0.10.1) (sh) >>> 0 73474 69974 5 68 0 6684 3740 wait IJ 0 0:00.02 | | = | `-- /usr/bin/make -C /usr/ports/graphics/py-graphviz FLAVOR=3Dpy311 = run-depends >>> 0 73496 73474 6 68 0 5584 2992 wait IJ 0 0:00.01 | | = | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh >>> 0 73521 73496 7 68 0 11144 8864 wait IJ 0 0:00.00 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>> 0 73522 73521 0 20 0 52432 32664 select SJ 0 0:02.70 | | = | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>> 0 76540 73522 3 68 0 26292 23812 urdlck IJ 0 0:00.02 | | = | `-- /usr/local/bin/dot -c >>> 0 91907 91496 5 68 0 15760 4492 nanslp S 0 1:05.17 | | = |-- sh: poudriere[main-armv7-poud-default]: html_json_main (sh) >>> 0 99134 91496 1 40 0 15760 4740 piperd I 0 0:03.22 | | = `-- sh: poudriere[main-armv7-poud-default]: pkg_cacher_main (sh) >>> 0 23 >=20 >=20 > A little bit more context for /usr/local/bin/dot : >=20 > 0x20631520 in _umtx_op () from /lib/libsys.so.7 > (gdb) bt > #0 0x20631520 in _umtx_op () at /lib/libsys.so.7 > #1 0x2063245c in _umtx_op_err () at /lib/libsys.so.7 > #2 0x203a2da8 in ??? () at /lib/libthr.so.3 > #3 0x2039bbf4 in ??? () at /lib/libthr.so.3 > #4 0x20061788 in ??? () at /libexec/ld-elf.so.1 >=20 > And the associated instance of /usr/local/sbin/pkg-static : >=20 > (gdb) bt > #0 _poll () at _poll.S:4 > #1 0x007669e0 in __thr_poll (fds=3D0xd1, nfds=3D1, timeout=3D1000) at = /home/pkgbuild/worktrees/main/lib/libthr/thread/thr_syscalls.c:320 > #2 0x003602e8 in pkg_script_run_child (pid=3D64019, = pstat=3Dpstat@entry=3D0xffffc49c, inputfd=3D9, script_name=3D0x5d5c9 = "POST-INSTALL") at scripts.c:303 > #3 0x0035fc34 in pkg_script_run (pkg=3D0x20972e00, type=3D, upgrade=3D) at scripts.c:227 > #4 0x00371250 in pkg_add_common (db=3D, = path=3D, path@entry=3D0xffffda7f = "/packages/All/graphviz-9.0.0_4.pkg", flags=3D, = reloc=3D, remote=3D0x0, local=3D0x0, t=3D0x0) at = pkg_add.c:1386 > #5 0x003707e4 in pkg_add (db=3D0x4, path=3D0x1 , path@entry=3D0xffffda7f = "/packages/All/graphviz-9.0.0_4.pkg", flags=3D1000, = location=3D0x766990 <__thr_poll> "\360H-\351\020\260\215\342") at = pkg_add.c:1460 > #6 0x00194544 in exec_add (argc=3D, argv=3D) at add.c:178 > #7 0x0019f840 in main (argc=3D2, argv=3D0xffffd87c) at main.c:872 >=20 >=20 Continued experiments point in a different direction via an simpler test of just use of "dot -c": # /usr/local/bin/dot -c Error: /usr/local/lib/graphviz/config6 is zero sized. And after the Error: line dot is hung up like it is when used via pkg-static. It seems that whatever leads to the "Error:" line output and that conidition's handling in dot is the source of the hangup. "dot -c" generates configuration file content for plugins and, appearently, should not gnerate an empty config6 file. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Thu Jul 18 08:14:41 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WPlwL6xDRz5R0Rt for ; Thu, 18 Jul 2024 08:14:58 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic304-23.consmr.mail.gq1.yahoo.com (sonic304-23.consmr.mail.gq1.yahoo.com [98.137.68.204]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4WPlwK6F2Gz4Ts7 for ; Thu, 18 Jul 2024 08:14:57 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=Xu6FoAoX; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.68.204 as permitted sender) smtp.mailfrom=marklmi@yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1721290496; bh=HHJimE+Lxxjym5t/awxTdBY9DdY/dSKPcWEu3pvxo28=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=Xu6FoAoXsbSMYGcYpb6yXhNnrj1WxmWLX7kmYvpmizB4lzliuMDbI/wGePnMbYSr+7ht/JThMWGorcxW5JwjPNQ4LSHncjm44JG814taDMkRnmbkGzvmcUkezba3q+N95EArdvAqilDT/iUwy5x/5k91BlI8+BmVCRE8HcYqF8vvb8PFrOBQ2iTa0QWZ6/JU1nraZ3twHfHL0z57QHQ4OChfHu1Gkc7j5sh1Bj5qtCNiwPEBpH0KRvKso0lDy34jf1Cxq5mKUzDhCXAgRWFDalmJkC3qTGIsDtwsYTcTWQG0vQK73HA/X0hAjdrJaH6OKLys8WDJS92EPHqcsmX+Wg== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1721290496; bh=lB7bgZzZOAyfdaMgCg5VfuDZfvXL5LHOGbAutN+IY8N=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=HhOn/Sgh3ag+qNb8ri+7PrxrGhGhDUjVLWO3ly/RqA/CykwMpweUk6a66x55jdUrRiKQQ7kv64NP0QODjqFYZIoXrOlf3pvr7QNN6YwF46Uz/4ZvUoSueSx+oQ0JTgxm1zBRDhZJUyUkooTEVmymSf25YTa0u6z/n7hb4d2ZmOMSfLiW1z0V8HliMO2bXiGB4kRmS9QhWqaP/k/FmVHrqFW3cNZVNyMmk9zSqiQPnmCjrhKJltvi86LCnpFoTHvBU2XzLy98FARNh/33kjbBT72XaCUsLI7fQr1r214uEq1gAH5O64y8+K25q77GLgAFUu/yV7LQK68xHtmA7Odnow== X-YMail-OSG: 9hHrW8EVM1mMZ4X3T_RVQ4o3ATCv1M65ocRjmYQ5YXEGioqUG93ByGvkWYkYeEn lStyjwr_UJdWEDPXxEU.K9bQgyQPsvCZqTaSzHseLohVGz9LsRNFx.eA_yZ6iCdBLgwMWgwY4fCn zqnoaHbI3COEY39TzehA4xzy7s2GrrY..cgLSeRuAD82F.8oFR7zAxEdug1T9Js767Zaw0TGLeQA 1qXk6PRBBhkFN.YFbraAlPD6h4sjY60EIdbXgTE8IiXN9Z8TcUT8wgo7E5__ScxvVmrtu2ZD2MV0 EcwMVFPXdsJYEG0jUj4B0s6P7qFc1LOqbFKAMIT6P.4CKcAXMw4v2OA7mfzFl.Yf6dl0uS0roSlB ximH_rIdRlgj2kTOBlcAfpTqSpoYDAmS6_vmF9XJASmEgh1W5dgu7e5c9KAJ.OIzl9pVMoL22S8U ND0qjP3qAJPWQ0C0moUaeRe1F6J_4VA3Sks3yUZhdh5mPfDx2EmGHD_kz.9yBXxH6cODwHT.VJ15 ZaruC5ovp.Zygx.NfaW6UOBf8Yh4z5TusZRDDariWNJtZsnpTQ.a0YVt27aGI42Q0TLq.6DoXV23 r0eL3oOeph2kZZ8JkV.vu0bZMWEoDsbiyNv1GMfg81DZcRqcHO1JGbZgRQOz1eLcUQolUmkghKtA 1AtPZjnb9dl0YrOO44QMnhi0w9wV8doNxllEJlmFqbFoADJQbDxnA9m8nXt1dpX_UydSDqj0I8kr vpJJivOImBqChiXWbjtX_DfCzUHosi26o8RewuvONbVW7jwse.WutpkCo0YQdVJUmLW3TQ1o7L7D ctTHGSTbmrZGH0.NVHLq87KknxlRrMnWEVK6q64X6Nj9Qz_7_odn9w4KhkdYUGcWHKKoPJ7nyb0y 8M96J.JcwCJEam62ZC8ja85zt2yQV7z2z4TXJki12erjibNQ97Z..QBz0tnnU9T7vweXWVR9JNa5 h1kwN7Gj6vf4kgWoz3t3S9jnouR47So2jl84pVq9ROnSB7i6RBGQ.pTZF1r3KJC2a0CfY2MT3VzL OVfYX3jTpPq7QW4ojDm9GeBjLizwvyx.siUR0nbi24FM7PkO4Q43htue1yLnFslViHojHFQ5V8Fw M2CXCAn37A5xJkgwo5KpwSDklfZb0dPvBZme2GhJNQ5oRDnjAmJqoX7me9x1YX2AcU..ct3HW8dm r.iigIFaIrFt_ibL70qJaHJ5KEn1hXXtAU6mgC_IRonSqGKmbKJEgIHPZ07ZxfLHQZaY8oeIM6m1 uf4OvGRCKpgRn1vMxebXIXvTdoc79.YijFIfZGnYIya5gFcETW_iKYGyNctZrWT9X6AIpiGOGrV_ _URLIRrwjJ6RGlavHL9zU2xHLEaNJnkyDR2cIhjb6svuJ4N4sjCEXzNjHHnDAu30.wxh6HZC2D_x UN19Ihjoe.2w4nVuvcSl9dbYI4I_B0Nh1UPTsPeP1Qo6Jn90swFF2wvot.Ee20qXcDajVylQ0Eua tzW.mOVQ3WS9fN7vSUtEPhhNwA9nOIX4wkgkVhDPUV8JllELrUOTPWdsDkq6HBP7oK8I6xKKF.nC EpggrfVLoSNz9zNzrZha_tnoWsUYZSO9_XCzMjYCmBLtgsASLbRj93ezraiPG7S9efwZ1rQh_eI8 uPkHT3lgH_ZQ7MP1p8Bt1hCWYv6JIPTKZfs7l46yzXTJ_pXGqSkc_Ci75SHlcueOhfSKCqPkTD4e ScDUamClMHvvpFcqi2PqFtfh0fEaW4hg.umAhpJAjEmgcUKfBtTpQuixlCbxmJByj0rs.rBmbbH2 1xIap6p6sZZiKLz4Dk2gIRwZ189.Lo0v1U_JrYrGA85rzempc5xEHmeAj7.U9aBnp35ASGqTXDde MGida4yTBSQ6qZX2m7IwNeLVWzE9Ns_wGjfW5y1VsqJzH9xRpOcqK5tBPaezDsHA9x.MCPJZXF64 49d9nhxhoZy9Z.K26cefozoUxswRoeQ_mUSipa4TUU480QhtWjUfabmAlhW2VoTQNC9EpauoEs8l 9VZj1AMGoD46H3nSl0b9YhOvaj4nhRbnnOSgP78IY6448TD25DeWrjsv_QYwr_ZxfS3q1HWjb2w8 QUBL45adDiEIeGFgVCis1d_WZz2fFAC8cuPb7BL.q2qi117NxT4RUC.0Tuho9miZjQFiLf8_JGBH Oi3.Al02bGpP8yFrjk6ky7CYv7M7Pwxzw9.NRxCgK9KfN1hr.t0sL3XT8_U1egnDAwSkVBR2gTCj zAhFTkQpuT33hPWiGrkGTbj96rdwqwlJbipD2AsJw9Lgdgzlk1ST2ezK9VgkwP1gAmMK6pETiJ2Q lpWengp79f7p8 X-Sonic-MF: X-Sonic-ID: 1af53bd8-be5f-4fbe-b3b3-190377466973 Received: from sonic.gate.mail.ne1.yahoo.com by sonic304.consmr.mail.gq1.yahoo.com with HTTP; Thu, 18 Jul 2024 08:14:56 +0000 Received: by hermes--production-gq1-799bb7c8cf-jmg5m (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 0482005562ff59ac810d772f04076ae7; Thu, 18 Jul 2024 08:14:52 +0000 (UTC) Content-Type: text/plain; charset=utf-8 List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.600.62\)) Subject: Re: armv7-on-aarch64 stuck at urdlck: I got a replication of the "ampere2" bulk build hangup problem on a Windows DevKit 2023 From: Mark Millard In-Reply-To: Date: Thu, 18 Jul 2024 01:14:41 -0700 Cc: Konstantin Belousov , Baptiste Daroussin Content-Transfer-Encoding: quoted-printable Message-Id: References: <8214703E-AB28-4FB3-A3DD-03C87363D8C6@yahoo.com> To: "olce@freebsd.org" , Philip Paeps , FreeBSD ARM List , FreeBSD Mailing List , Current FreeBSD X-Mailer: Apple Mail (2.3774.600.62) X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.84 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.84)[-0.841]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; DKIM_TRACE(0.00)[yahoo.com:+]; FREEMAIL_FROM(0.00)[yahoo.com]; TO_DN_SOME(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MLMMJ_DEST(0.00)[freebsd-arm@freebsd.org]; RCPT_COUNT_SEVEN(0.00)[7]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.68.204:from]; APPLE_MAILER_COMMON(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.68.204:from] X-Rspamd-Queue-Id: 4WPlwK6F2Gz4Ts7 On Jul 16, 2024, at 23:45, Mark Millard wrote: > On Jul 16, 2024, at 18:41, Mark Millard wrote: >=20 >> On Jul 16, 2024, at 11:37, Mark Millard wrote: >>=20 >>> On Jul 16, 2024, at 10:42, Mark Millard wrote: >>>=20 >>>> No longer is the problem only observed on ampere2! But this was = with >>>> a non-debug, personally built kernel that has some of my now = patches. >>>> I'll see if I can replicate the issue with an official pkgbase = debug >>>> kernel. >>>=20 >>> It replicated with the official pkgbase debug kernel. The >>> kernel did not report anything. >>>=20 >>> The following commits in main and happen between the last working >>> ampere2 armv7 builds and the first failing ampere2 builds and >>> look be the only likely contributors from that range as far as >>> I could tell: >>>=20 >>> Tue, 27 Feb 2024 >>> . . . >>> =E2=80=A2 git: 1df8700aa6cf - main - PP mutexes: unlock: Reset = inherited prio regardless of privileges Olivier Certner=20 >>> =E2=80=A2 git: 9ac3ac9ece62 - main - PP mutexes: lock: Check if = priority is too high against base one Olivier Certner=20 >>> =E2=80=A2 git: 39e4665c9694 - main - PP mutexes: lock: Reduce = 'umtx_lock' holding before taking the user lock Olivier Certner >>>=20 >>> These changes are not in 14.0-RELEASE but are in 14.1-STABLE and = 14.1-RELEASE. >>> So I expect that when any ampere*'s progress to 14.1-RELEASE the = armv7 >>> problems would start for them. >>>=20 >>> These changes are not in 13.3-RELEASE but are in 13.3-STABLE. So I = expect that >>> when any ampere*'s progress to 13.4-RELEASE the problems would start = for them. >>>=20 >>>=20 >>> With the prior packages already built in a prior poudriere-devel >>> run it turns out that just: >>>=20 >>> # poudriere bulk -j main-armv7-poud -i graphics/graphviz >>>=20 >>> replicates the problem: >>>=20 >>> . . . >>> [00:00:45] Installing graphics/graphviz | graphviz-9.0.0_4 >>> [aarch64PBase] Installing graphviz-9.0.0_4... >>> [aarch64PBase] `-- Installing cairo-1.17.4_2,3... >>> [aarch64PBase] | `-- Installing fontconfig-2.15.0_2,1... >>> [aarch64PBase] | | `-- Installing expat-2.6.2... >>> [aarch64PBase] | | `-- Extracting expat-2.6.2: 100% >>> [aarch64PBase] | | `-- Installing freetype2-2.13.2... >>> [aarch64PBase] | | `-- Installing brotli-1.1.0,1... >>> [aarch64PBase] | | `-- Extracting brotli-1.1.0,1: 100% >>> [aarch64PBase] | | `-- Installing png-1.6.43... >>> [aarch64PBase] | | `-- Extracting png-1.6.43: 100% >>> [aarch64PBase] | | `-- Extracting freetype2-2.13.2: 100% >>> [aarch64PBase] | `-- Extracting fontconfig-2.15.0_2,1: 100% >>> [aarch64PBase] | `-- Installing glib-2.80.4,2... >>> [aarch64PBase] | | `-- Installing libffi-3.4.6... >>> [aarch64PBase] | | `-- Extracting libffi-3.4.6: 100% >>> [aarch64PBase] | | `-- Installing libiconv-1.17_1... >>> [aarch64PBase] | | `-- Extracting libiconv-1.17_1: 100% >>> [aarch64PBase] | | `-- Installing pcre2-10.43... >>> [aarch64PBase] | | `-- Extracting pcre2-10.43: 100% >>> [aarch64PBase] | | `-- Installing py311-packaging-24.1... >>> [aarch64PBase] | | `-- Installing python311-3.11.9_1... >>> [aarch64PBase] | | | `-- Installing mpdecimal-4.0.0... >>> [aarch64PBase] | | | `-- Extracting mpdecimal-4.0.0: 100% >>> [aarch64PBase] | | | `-- Installing readline-8.2.10... >>> [aarch64PBase] | | | `-- Extracting readline-8.2.10: 100% >>> [aarch64PBase] | | `-- Extracting python311-3.11.9_1: 100% >>> [aarch64PBase] | | `-- Extracting py311-packaging-24.1: 100% >>> [aarch64PBase] | `-- Extracting glib-2.80.4,2: 100% >>> [aarch64PBase] | `-- Installing libglvnd-1.7.0... >>> [aarch64PBase] | `-- Extracting libglvnd-1.7.0: 100% >>> [aarch64PBase] | `-- Installing pixman-0.42.2... >>> [aarch64PBase] | `-- Extracting pixman-0.42.2: 100% >>> [aarch64PBase] `-- Extracting cairo-1.17.4_2,3: 100% >>> [aarch64PBase] `-- Installing harfbuzz-9.0.0... >>> [aarch64PBase] | `-- Installing graphite2-1.3.14... >>> [aarch64PBase] | `-- Extracting graphite2-1.3.14: 100% >>> [aarch64PBase] `-- Extracting harfbuzz-9.0.0: 100% >>> [aarch64PBase] `-- Installing jpeg-turbo-3.0.3... >>> [aarch64PBase] `-- Extracting jpeg-turbo-3.0.3: 100% >>> [aarch64PBase] `-- Installing libgd-2.3.3_13,1... >>> [aarch64PBase] | `-- Installing tiff-4.6.0... >>> [aarch64PBase] | | `-- Installing jbigkit-2.1_3... >>> [aarch64PBase] | | `-- Extracting jbigkit-2.1_3: 100% >>> [aarch64PBase] | | `-- Installing lerc-4.0.0... >>> [aarch64PBase] | | `-- Extracting lerc-4.0.0: 100% >>> [aarch64PBase] | | `-- Installing libdeflate-1.20... >>> [aarch64PBase] | | `-- Extracting libdeflate-1.20: 100% >>> [aarch64PBase] | | `-- Installing zstd-1.5.6... >>> [aarch64PBase] | | `-- Installing liblz4-1.9.4_1,1... >>> [aarch64PBase] | | `-- Extracting liblz4-1.9.4_1,1: 100% >>> [aarch64PBase] | | `-- Extracting zstd-1.5.6: 100% >>> [aarch64PBase] | `-- Extracting tiff-4.6.0: 100% >>> [aarch64PBase] | `-- Installing webp-1.4.0_1... >>> [aarch64PBase] | | `-- Installing giflib-5.2.2... >>> [aarch64PBase] | | `-- Extracting giflib-5.2.2: 100% >>> [aarch64PBase] | `-- Extracting webp-1.4.0_1: 100% >>> [aarch64PBase] `-- Extracting libgd-2.3.3_13,1: 100% >>> [aarch64PBase] `-- Installing libltdl-2.4.7... >>> [aarch64PBase] `-- Extracting libltdl-2.4.7: 100% >>> [aarch64PBase] `-- Installing pango-1.52.2_1... >>> [aarch64PBase] | `-- Installing fribidi-1.0.15... >>> [aarch64PBase] | `-- Extracting fribidi-1.0.15: 100% >>> [aarch64PBase] | `-- Installing libXft-2.3.8... >>> [aarch64PBase] | `-- Extracting libXft-2.3.8: 100% >>> [aarch64PBase] | `-- Installing libthai-0.1.29_1... >>> [aarch64PBase] | | `-- Installing libdatrie-0.2.13_2... >>> [aarch64PBase] | | `-- Extracting libdatrie-0.2.13_2: 100% >>> [aarch64PBase] | `-- Extracting libthai-0.1.29_1: 100% >>> [aarch64PBase] `-- Extracting pango-1.52.2_1: 100% >>> [aarch64PBase] Extracting graphviz-9.0.0_4: 100% >>>=20 >>> And here it is hung with /usr/local/bin/dot -c in urdlck : >>>=20 >>> 0 1483 4502 7 68 0 15760 4872 wait I+ 0 0:03.92 | | = `-- /usr/local/libexec/poudriere/sh -e -o pipefail = /usr/local/share/poudriere/bulk.sh -j main-armv7-poud -i = graphics/graphviz >>> 0 1894 1483 5 68 0 15760 4712 nanslp S 0 0:02.07 | | = |-- sh: poudriere[main-armv7-poud-default]: html_json_main (sh) >>> 0 25321 1483 6 68 0 6664 3868 wait I+J 0 0:00.11 | | = `-- /usr/bin/make -C /usr/ports/graphics/graphviz install-package >>> 0 25322 25321 5 68 0 11140 8860 wait I+J 0 0:00.00 | | = `-- /usr/local/sbin/pkg-static add = /packages/All/graphviz-9.0.0_4.pkg >>> 0 25323 25322 5 20 0 63824 45144 select S+J 0 0:02.85 | | = `-- /usr/local/sbin/pkg-static add = /packages/All/graphviz-9.0.0_4.pkg >>> 0 26900 25323 3 68 0 26292 23804 urdlck I+J 0 0:00.02 | | = `-- /usr/local/bin/dot -c >>>=20 >>>=20 >>>> FYI for the replication that I got: >>>>=20 >>>> /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>>> runs: >>>> /usr/local/bin/dot -c >>>>=20 >>>> each such /usr/local/bin/dot is stuck at MWCHAN urdlck . >>>>=20 >>>> # poudriere status -b >>>> [main-armv7-poud-default] [2024-07-16_04h27m31s] [parallel_build] = Queued: 449 Built: 433 Failed: 0 Skipped: 0 Ignored: 0 Fetched: 0 = Tobuild: 16 Time: 04:44:38 >>>> ID TOTAL ORIGIN PKGNAME = PHASE PHASE TMPFS CPU% MEM% >>>> [01] 00:59:45 graphics/rubygem-ruby-graphviz | = rubygem-ruby-graphviz-1.2.5 run-depends 00:59:30 1.59 GiB 0% 0.2% >>>> [02] 00:49:59 graphics/p5-GraphViz | p5-GraphViz-2.25 = build-depends 00:49:53 1.45 GiB 0% 0.2% >>>> [03] 00:59:45 graphics/py-pydot@py311 | py311-pydot-2.0.0 = run-depends 00:59:34 1.47 GiB 0% 0.2% >>>> [04] 00:59:45 graphics/py-pygraphviz@py311 | py311-pygraphviz-1.6 = lib-depends 00:59:33 1.47 GiB 0% 0.2% >>>> [05] 00:58:57 graphics/py-graphviz@py311 | = py311-graphviz-0.10.1 run-depends 00:58:49 1.47 GiB 0% 0.2% >>>> [06] 00:59:20 audio/ganv | ganv-1.8.2_1 = lib-depends 00:59:10 1.53 GiB 0% 0.2% >>>> [07] 00:59:45 devel/libr3 | libr3-1.0.0_2 = lib-depends 00:59:19 1.54 GiB 0% 0.3% >>>> [08] 00:59:45 net/netmap | netmap-0.1.3_2 = run-depends 00:59:22 1.46 GiB 0% 0.3% >>>>=20 >>>> I had started the bulk build via the list: >>>>=20 >>>> # more ~/origins/ampere2-failures-armv7.txt >>>> audio/ganv >>>> devel/doxygen >>>> devel/libr3 >>>> graphics/p5-GraphViz >>>> graphics/p5-GraphViz2 >>>> graphics/oyranos >>>> graphics/pear-Image_GraphViz@php81 >>>> graphics/py-graphviz@py311 >>>> graphics/py-pydot@py311 >>>> graphics/py-pygraphviz@py311 >>>> graphics/rubygem-ruby-graphviz >>>> math/ggobi >>>> net-mgmt/librenms >>>> net/netmap >>>> print/dot2tex@py311 >>>>=20 >>>> # poudriere bulk -j main-armv7-poud `cat = ~/origins/ampere2-failures-armv7.txt` >>>> . . . >>>> [00:00:12] Building 449 packages using up to 8 builders >>>> . . . >>>> [03:44:55] [01] [00:18:54] Finished graphics/graphviz | = graphviz-9.0.0_4: Success >>>> [03:44:56] [01] [00:00:00] Building = graphics/rubygem-ruby-graphviz | rubygem-ruby-graphviz-1.2.5 >>>> [03:44:56] [03] [00:00:00] Building graphics/py-pydot@py311 | = py311-pydot-2.0.0 >>>> [03:44:56] [04] [00:00:00] Building graphics/py-pygraphviz@py311 = | py311-pygraphviz-1.6 >>>> [03:44:56] [07] [00:00:00] Building devel/libr3 | libr3-1.0.0_2 >>>> [03:44:56] [08] [00:00:00] Building net/netmap | netmap-0.1.3_2 >>>> [03:45:21] [06] [00:09:33] Finished x11-toolkits/gtkmm24 | = gtkmm24-2.24.5_4: Success >>>> [03:45:21] [06] [00:00:00] Building audio/ganv | ganv-1.8.2_1 >>>> [03:45:44] [05] [00:13:33] Finished graphics/ImageMagick6@nox11 | = ImageMagick6-nox11-6.9.12.77_9,1: Success >>>> [03:45:44] [05] [00:00:00] Building graphics/py-graphviz@py311 | = py311-graphviz-0.10.1 >>>> [03:54:42] [02] [00:24:53] Finished print/texlive-base | = texlive-base-20240312: Success >>>> [03:54:42] [02] [00:00:00] Building graphics/p5-GraphViz | = p5-GraphViz-2.25 >>>>=20 >>>> In /usr/src/sys/kern/kern_umtx.c there is: >>>>=20 >>>> static int >>>> do_rw_rdlock(struct thread *td, struct urwlock *rwlock, long fflag, >>>> struct _umtx_time *timeout) >>>> { >>>> . . . >>>> /* >>>> * Contention bit is set, before sleeping, increase >>>> * read waiter count. >>>> */ >>>> rv =3D fueword32(&rwlock->rw_blocked_readers, >>>> &blocked_readers); >>>> if (rv =3D=3D 0) >>>> rv =3D suword32(&rwlock->rw_blocked_readers, >>>> blocked_readers + 1); >>>> if (rv =3D=3D -1) { >>>> umtxq_unbusy_unlocked(&uq->uq_key); >>>> error =3D EFAULT; >>>> break; >>>> } >>>> while (state & wrflags) { >>>> umtxq_lock(&uq->uq_key); >>>> umtxq_insert(uq); >>>> umtxq_unbusy(&uq->uq_key); >>>> error =3D umtxq_sleep(uq, "urdlck", timeout = =3D=3D NULL ? >>>> NULL : &timo); >>>> umtxq_busy(&uq->uq_key); >>>> umtxq_remove(uq); >>>> umtxq_unlock(&uq->uq_key); >>>> if (error) >>>> break; >>>> rv =3D fueword32(&rwlock->rw_state, &state); >>>> if (rv =3D=3D -1) { >>>> error =3D EFAULT; >>>> break; >>>> } >>>> } >>>>=20 >>>> . . . >>>>=20 >>>>=20 >>>>=20 >>>> For reference: >>>>=20 >>>> # ps -alxdww | less >>>> UID PID PPID C PRI NI VSZ RSS MWCHAN STAT TT TIME = COMMAND >>>> . . . >>>> 0 87700 4522 6 20 0 16576 1888 - T 0 0:00.01 | = | |-- vi /usr/local/share/poudriere/jail.sh >>>> 0 91496 4522 4 20 0 15760 4684 select S+ 0 0:06.88 | = | `-- /usr/local/libexec/poudriere/sh -e -o pipefail = /usr/local/share/poudriere/bulk.sh audio/ganv devel/doxygen devel/libr3 = graphics/p5-GraphViz graphics/p5-GraphViz2 graphics/oyranos = graphics/pear-Image_GraphViz@php81 graphics/py-graphviz@py311 = graphics/py-pydot@py311 graphics/py-pygraphviz@py311 = graphics/rubygem-ruby-graphviz math/ggobi net-mgmt/librenms net/netmap = print/dot2tex@py311 >>>> 0 37688 91496 0 68 0 15760 4700 wait I 0 0:00.05 | = | |-- sh: poudriere[main-armv7-poud-default][01]: build_pkg = (rubygem-ruby-graphviz-1.2.5) (sh) >>>> 0 47568 37688 0 68 0 6664 3664 wait IJ 0 0:00.03 | = | | `-- /usr/bin/make -C /usr/ports/graphics/rubygem-ruby-graphviz = run-depends >>>> 0 47598 47568 6 68 0 5568 2988 wait IJ 0 0:00.01 | = | | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh >>>> 0 47743 47598 6 68 0 11188 8864 wait IJ 0 0:00.00 | = | | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>>> 0 47747 47743 3 20 0 71692 48984 select SJ 0 0:04.26 | = | | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>>> 0 56383 47747 1 68 0 26292 23812 urdlck IJ 0 0:00.02 | = | | `-- /usr/local/bin/dot -c >>>> 0 37700 91496 6 68 0 15760 4700 wait I 0 0:00.04 | = | |-- sh: poudriere[main-armv7-poud-default][03]: build_pkg = (py311-pydot-2.0.0) (sh) >>>> 0 45102 37700 2 68 0 6668 3704 wait IJ 0 0:00.02 | = | | `-- /usr/bin/make -C /usr/ports/graphics/py-pydot FLAVOR=3Dpy311 = run-depends >>>> 0 45156 45102 4 68 0 5584 2992 wait IJ 0 0:00.01 | = | | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh >>>> 0 45215 45156 4 68 0 11144 8864 wait IJ 0 0:00.00 | = | | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>>> 0 45218 45215 4 20 0 51420 31512 select SJ 0 0:02.68 | = | | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>>> 0 52147 45218 2 68 0 26292 23812 urdlck IJ 0 0:00.02 | = | | `-- /usr/local/bin/dot -c >>>> 0 37721 91496 1 68 0 15760 4700 wait I 0 0:00.04 | = | |-- sh: poudriere[main-armv7-poud-default][04]: build_pkg = (py311-pygraphviz-1.6) (sh) >>>> 0 45937 37721 1 68 0 6684 3744 wait IJ 0 0:00.03 | = | | `-- /usr/bin/make -C /usr/ports/graphics/py-pygraphviz = FLAVOR=3Dpy311 lib-depends >>>> 0 46009 45937 7 68 0 5584 2992 wait IJ 0 0:00.01 | = | | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh >>>> 0 46127 46009 7 68 0 11144 8864 wait IJ 0 0:00.00 | = | | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>>> 0 46129 46127 1 20 0 51384 31548 select SJ 0 0:02.73 | = | | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>>> 0 53311 46129 4 68 0 26292 23812 urdlck IJ 0 0:00.02 | = | | `-- /usr/local/bin/dot -c >>>> 0 37744 91496 7 45 0 15760 4692 wait I 0 0:00.04 | = | |-- sh: poudriere[main-armv7-poud-default][07]: build_pkg = (libr3-1.0.0_2) (sh) >>>> 0 55198 37744 0 50 0 6664 3664 wait IJ 0 0:00.03 | = | | `-- /usr/bin/make -C /usr/ports/devel/libr3 lib-depends >>>> 0 55229 55198 0 68 0 5588 2988 wait IJ 0 0:00.01 | = | | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh >>>> 0 55594 55229 7 68 0 11168 8864 wait IJ 0 0:00.00 | = | | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>>> 0 55596 55594 2 20 0 69796 50180 select SJ 0 0:04.53 | = | | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>>> 0 62753 55596 0 68 0 26292 23812 urdlck IJ 0 0:00.02 | = | | `-- /usr/local/bin/dot -c >>>> 0 37763 91496 4 29 0 15760 4696 wait I 0 0:00.05 | = | |-- sh: poudriere[main-armv7-poud-default][08]: build_pkg = (netmap-0.1.3_2) (sh) >>>> 0 51054 37763 6 36 0 6636 3684 wait IJ 0 0:00.03 | = | | `-- /usr/bin/make -C /usr/ports/net/netmap run-depends >>>> 0 51107 51054 3 68 0 5568 2988 wait IJ 0 0:00.01 | = | | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh >>>> 0 51576 51107 3 68 0 11168 8860 wait IJ 0 0:00.00 | = | | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>>> 0 51580 51576 3 20 0 68220 49432 select SJ 0 0:04.27 | = | | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>>> 0 59063 51580 6 68 0 26292 23808 urdlck IJ 0 0:00.02 | = | | `-- /usr/local/bin/dot -c >>>> 0 53709 91496 1 68 0 15760 4700 wait I 0 0:00.04 | = | |-- sh: poudriere[main-armv7-poud-default][06]: build_pkg = (ganv-1.8.2_1) (sh) >>>> 0 63371 53709 6 68 0 6636 3668 wait IJ 0 0:00.03 | = | | `-- /usr/bin/make -C /usr/ports/audio/ganv lib-depends >>>> 0 63377 63371 1 68 0 5580 2996 wait IJ 0 0:00.01 | = | | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh >>>> 0 63413 63377 4 68 0 11180 8864 wait IJ 0 0:00.00 | = | | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>>> 0 63414 63413 3 20 0 56212 35980 select SJ 0 0:02.21 | = | | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>>> 0 65577 63414 6 68 0 26292 23812 urdlck IJ 0 0:00.01 | = | | `-- /usr/local/bin/dot -c >>>> 0 63365 91496 3 68 0 15760 4696 wait I 0 0:00.03 | = | |-- sh: poudriere[main-armv7-poud-default][02]: build_pkg = (p5-GraphViz-2.25) (sh) >>>> 0 63807 63365 7 68 0 6696 3672 wait IJ 0 0:00.02 | = | | `-- /usr/bin/make -C /usr/ports/graphics/p5-GraphViz = build-depends >>>> 0 63808 63807 2 68 0 5568 2988 wait IJ 0 0:00.01 | = | | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh >>>> 0 63833 63808 4 68 0 11188 8864 wait IJ 0 0:00.00 | = | | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>>> 0 63834 63833 7 20 0 67400 48532 select SJ 0 0:03.52 | = | | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>>> 0 66041 63834 2 68 0 26292 23812 urdlck IJ 0 0:00.02 | = | | `-- /usr/local/bin/dot -c >>>> 0 69974 91496 1 68 0 15760 4700 wait I 0 0:00.04 | = | |-- sh: poudriere[main-armv7-poud-default][05]: build_pkg = (py311-graphviz-0.10.1) (sh) >>>> 0 73474 69974 5 68 0 6684 3740 wait IJ 0 0:00.02 | = | | `-- /usr/bin/make -C /usr/ports/graphics/py-graphviz = FLAVOR=3Dpy311 run-depends >>>> 0 73496 73474 6 68 0 5584 2992 wait IJ 0 0:00.01 | = | | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh >>>> 0 73521 73496 7 68 0 11144 8864 wait IJ 0 0:00.00 | = | | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>>> 0 73522 73521 0 20 0 52432 32664 select SJ 0 0:02.70 | = | | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>>> 0 76540 73522 3 68 0 26292 23812 urdlck IJ 0 0:00.02 | = | | `-- /usr/local/bin/dot -c >>>> 0 91907 91496 5 68 0 15760 4492 nanslp S 0 1:05.17 | = | |-- sh: poudriere[main-armv7-poud-default]: html_json_main (sh) >>>> 0 99134 91496 1 40 0 15760 4740 piperd I 0 0:03.22 | = | `-- sh: poudriere[main-armv7-poud-default]: pkg_cacher_main (sh) >>>> 0 23 >>=20 >>=20 >> A little bit more context for /usr/local/bin/dot : >>=20 >> 0x20631520 in _umtx_op () from /lib/libsys.so.7 >> (gdb) bt >> #0 0x20631520 in _umtx_op () at /lib/libsys.so.7 >> #1 0x2063245c in _umtx_op_err () at /lib/libsys.so.7 >> #2 0x203a2da8 in ??? () at /lib/libthr.so.3 >> #3 0x2039bbf4 in ??? () at /lib/libthr.so.3 >> #4 0x20061788 in ??? () at /libexec/ld-elf.so.1 >>=20 >> And the associated instance of /usr/local/sbin/pkg-static : >>=20 >> (gdb) bt >> #0 _poll () at _poll.S:4 >> #1 0x007669e0 in __thr_poll (fds=3D0xd1, nfds=3D1, timeout=3D1000) = at /home/pkgbuild/worktrees/main/lib/libthr/thread/thr_syscalls.c:320 >> #2 0x003602e8 in pkg_script_run_child (pid=3D64019, = pstat=3Dpstat@entry=3D0xffffc49c, inputfd=3D9, script_name=3D0x5d5c9 = "POST-INSTALL") at scripts.c:303 >> #3 0x0035fc34 in pkg_script_run (pkg=3D0x20972e00, type=3D, upgrade=3D) at scripts.c:227 >> #4 0x00371250 in pkg_add_common (db=3D, = path=3D, path@entry=3D0xffffda7f = "/packages/All/graphviz-9.0.0_4.pkg", flags=3D, = reloc=3D, remote=3D0x0, local=3D0x0, t=3D0x0) at = pkg_add.c:1386 >> #5 0x003707e4 in pkg_add (db=3D0x4, path=3D0x1 , path@entry=3D0xffffda7f = "/packages/All/graphviz-9.0.0_4.pkg", flags=3D1000, = location=3D0x766990 <__thr_poll> "\360H-\351\020\260\215\342") at = pkg_add.c:1460 >> #6 0x00194544 in exec_add (argc=3D, argv=3D) at add.c:178 >> #7 0x0019f840 in main (argc=3D2, argv=3D0xffffd87c) at main.c:872 >>=20 >>=20 >=20 > Continued experiments point in a different direction > via an simpler test of just use of "dot -c": >=20 > # /usr/local/bin/dot -c > Error: /usr/local/lib/graphviz/config6 is zero sized. >=20 > And after the Error: line dot is hung up like it is > when used via pkg-static. >=20 > It seems that whatever leads to the "Error:" line > output and that conidition's handling in dot is the > source of the hangup. >=20 > "dot -c" generates configuration file content for > plugins and, appearently, should not gnerate an > empty config6 file. Well, I get to: (gdb) bt #0 0x2005acc0 in dlopen () from /libexec/ld-elf.so.1 #1 0x201b87fc in vm_open (loader_data=3D, = filename=3Dfilename@entry=3D0x20662540 = "/usr/local/lib/graphviz/libgvplugin_gd.so.6", advise=3D, = advise@entry=3D0x0) at loaders/dlopen.c:211 #2 0x201b6f24 in tryall_dlopen (phandle=3D, = phandle@entry=3D0xffffd978, filename=3D0x20662540 = "/usr/local/lib/graphviz/libgvplugin_gd.so.6", advise=3D0x0, vtable=3D0x0)= at ltdl.c:444 #3 0x201b52d0 in try_dlopen (phandle=3Dphandle@entry=3D0xffffd9b0, = filename=3D, filename@entry=3D0x20665040 = "/usr/local/lib/graphviz/libgvplugin_gd.so.6", ext=3D0x20662599 ".6", = advise=3D) at ltdl.c:1481 #4 0x201b4d34 in lt_dlopenadvise (filename=3D0x20665040 = "/usr/local/lib/graphviz/libgvplugin_gd.so.6", advise=3D0x0) at = ltdl.c:1671 #5 lt_dlopen (filename=3D0x1 ) at ltdl.c:1626 #6 0x200e255c in ?? () But the dlopen does not return. One possible point of interest is that /usr/local/lib/graphviz/libgvplugin_gd.so.6 leads to loading a bunch of libraries, including the first/only load of /lib/libc++.so.1 and libcxxrt.so.1 ( via /usr/local/lib/libLerc.so.4 ). =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sat Jul 20 00:04:55 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WQmy82dj8z5QfP0 for ; Sat, 20 Jul 2024 00:05:04 +0000 (UTC) (envelope-from arjun.anantharam@intel.com) Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.11]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mgamail.intel.com", Issuer "Sectigo RSA Organization Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WQmy667Z2z4h0s for ; Sat, 20 Jul 2024 00:05:02 +0000 (UTC) (envelope-from arjun.anantharam@intel.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=intel.com header.s=Intel header.b=Ncpvhy6d; dmarc=pass (policy=none) header.from=intel.com; spf=pass (mx1.freebsd.org: domain of arjun.anantharam@intel.com designates 198.175.65.11 as permitted sender) smtp.mailfrom=arjun.anantharam@intel.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1721433903; x=1752969903; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=q29Fc50kxpCtr1AMGB82Nz6KJa8y47uNevf6mTdyjbw=; b=Ncpvhy6dZf2o8mWmUCezHRoLoExVvJCGwE+4Ce4jykR0VbgkgCTBoPt3 oaPnTEw8hDVEkRm4GeVWemyjPPctqlEDrBf4S9HckrCPyI1M9hQ+DV9Wg Cua0XGd6toDsAzFmklrz6LCx4/rU5GsNQaZx0DwTaU2BJ9EhWNq+2e955 aEtv9iwnSGcJ8ZlZWxpJOT6yxIdHkPlgDF3YyPoD+pbn5wTblLVeaX1qc MWwTNTuKPmBotGpnYFZb410B6IaGtQWjLuDUzMBc4KGwvEMBIN5mnEnkd zyOFunwfjxjfaSoQD2SWkG3+WQeC50Lo0tOWYj9fGvCBEDqkK6+bxuzxH w==; X-CSE-ConnectionGUID: TV5RrnNkR3CaSifJSHrVyg== X-CSE-MsgGUID: Nu42PtPRS4uIHatNE4g/Sg== X-IronPort-AV: E=McAfee;i="6700,10204,11138"; a="29655836" X-IronPort-AV: E=Sophos;i="6.09,222,1716274800"; d="scan'208,217";a="29655836" Received: from orviesa003.jf.intel.com ([10.64.159.143]) by orvoesa103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Jul 2024 17:05:00 -0700 X-CSE-ConnectionGUID: gRz5onZaSfS67HMvRt8Auw== X-CSE-MsgGUID: jkqw6PXJQLmc2v7nXxAevw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.09,222,1716274800"; d="scan'208,217";a="56115268" Received: from fmsmsx602.amr.corp.intel.com ([10.18.126.82]) by orviesa003.jf.intel.com with ESMTP/TLS/AES256-GCM-SHA384; 19 Jul 2024 17:05:00 -0700 Received: from fmsmsx612.amr.corp.intel.com (10.18.126.92) by fmsmsx602.amr.corp.intel.com (10.18.126.82) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Fri, 19 Jul 2024 17:04:59 -0700 Received: from fmsedg601.ED.cps.intel.com (10.1.192.135) by fmsmsx612.amr.corp.intel.com (10.18.126.92) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39 via Frontend Transport; Fri, 19 Jul 2024 17:04:59 -0700 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (104.47.58.174) by edgegateway.intel.com (192.55.55.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.39; Fri, 19 Jul 2024 17:04:58 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=KJ4/JjCXUKGniZw04qwAWGITbo95xigPqEvaoYOmJ2D0LuuMfbvvKZ+zirk3ILFg/PUjM1mBTUejmjA+ywMyc+CfdlDZMpWCvqZ2TOyNigJn1jmSZ4EX1fymG+yDHMlySty0Z2oqm7Eg37xHtlXzaWKvzBeyxmdl+sEFwRTvlErroeJSLvItUSX4tCnx+fTzi7ULVzS4j7yc18zQ9wEFjZjoUK1fABXI5aJFHhTF/NTZ/pc5nR2YYlcaa/Nfyg1Izo+vnfSv08Vos8wF5klM4R6m6jyNGvzXgbwNI3Aegb5WApzku+Gy/nI7qsHU+2aBWKI50T83xlU4BHSjxcfhmA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=p9XJ1uP+eWUkp9L0z1LuMnVZmcIVGm4N6B6uhtnJjv8=; b=CtZlfYLE4g64hde+elOlq/+cQDW1vf8rEjeqjMMxKl1Y1CGZwWz/5wNjvoOQmP0tVBWuGhOylmfT07HXTG01RBkpsiKrHxwtaa/cR/INKyA7gS0clG9bhxPEgzZkFyUR01iO61JDaosgR9RSwxgILtsrpUGHG8+g33qLhNsfKFt31TWfcznJbMiBy6sY5R6CJftK5K/k0gQr+9dIWzN8LpFEpYTqd171iIInrbR9+KLb3zYUctx1Ib8oHTYmA0qM1HKNoO7gXoiis7nzcqBwlaupPkkZai0q9NkCc5ZicS5YKaYTgb0ERPBy/DcGTN3wQDAkHyzXYm38mQHrgcxulg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from MW3PR11MB4761.namprd11.prod.outlook.com (2603:10b6:303:53::15) by PH0PR11MB5807.namprd11.prod.outlook.com (2603:10b6:510:140::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7784.16; Sat, 20 Jul 2024 00:04:55 +0000 Received: from MW3PR11MB4761.namprd11.prod.outlook.com ([fe80::f9ab:2e1b:14d1:8737]) by MW3PR11MB4761.namprd11.prod.outlook.com ([fe80::f9ab:2e1b:14d1:8737%4]) with mapi id 15.20.7784.016; Sat, 20 Jul 2024 00:04:55 +0000 From: "Anantharam, Arjun" To: Andrew Turner CC: Warner Losh , Mark Millard , "freebsd-arm@freebsd.org" Subject: Re: FreeBSD OS installation stuck after selecting Boot installer Thread-Topic: FreeBSD OS installation stuck after selecting Boot installer Thread-Index: AQHaw5kHGGFRg6Nu5kenUR8xJXWTO7HRs0FpgAAEywCAAJ6dS4AAj+RzgAHvjhWABGU/MIAACxcAgAGBNamAAjfAAIAFj7N1gAAL5EqAAxKkbYAKj/A7gAArhQCADn79Mw== Date: Sat, 20 Jul 2024 00:04:55 +0000 Message-ID: References: <9BE6FB50-27DC-453E-B915-131D45C901CD.ref@yahoo.com> <9BE6FB50-27DC-453E-B915-131D45C901CD@yahoo.com> <1DEED962-C5D8-4B6D-8BC9-870A9BFE1176@fubar.geek.nz> <084E63D4-8A02-4E7B-8C26-152ED0D0317A@fubar.geek.nz> In-Reply-To: <084E63D4-8A02-4E7B-8C26-152ED0D0317A@fubar.geek.nz> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: x-ms-publictraffictype: Email x-ms-traffictypediagnostic: MW3PR11MB4761:EE_|PH0PR11MB5807:EE_ x-ms-office365-filtering-correlation-id: b4021be4-18c7-48f4-7715-08dca84f9603 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0;ARA:13230040|366016|1800799024|376014|38070700018; x-microsoft-antispam-message-info: =?Windows-1252?Q?2QiCUW2Stpjfhqmkyq5ASc+9ascyRaIwthKHOc03rc0zTp64sRhCfB8M?= =?Windows-1252?Q?yJZQFlelsW+K2LVVqwV+ED60MHpsNuIbIjaY/cK9tVymW+zBnrk0x8wJ?= =?Windows-1252?Q?yB/sQ9Y070HxrsOs/YjU4OROwePCXeeXoUEW+cuIlC9yMjwoTfWkzB2/?= =?Windows-1252?Q?Yty47DJv+7k7oX9ZLHxi7IdiSBSYwi2UsCv6Lw13lNe90E1kFuYJCVFn?= =?Windows-1252?Q?O44G7+UDB0QxA9raX0JNehSJ3IlViJHKNpJU9a87FnG7Pz881xyNEVY6?= =?Windows-1252?Q?30b41Za8pXy6zUNetSKk+crGdgNhMrHI6zhSDJruCHwPRHouj2eC5Gnj?= =?Windows-1252?Q?vD9U8XXnGw/Lx8u3DkZ/whLWRBOq6VhiMvI2Pk0rUZOh8CXlrWRW646A?= =?Windows-1252?Q?CiYDaaD/QPRIK5xTBT+AFuxsrjDrgiv26u+uycoP0nWU/D/yWWPn9WKF?= =?Windows-1252?Q?Fctx465UjKSlU4fVCbO69bgOyMQLrz3fic3v7rOxS4TnG53IcnOYgRHg?= =?Windows-1252?Q?vufFjvotKsgOdHF5QNmNo9RPF5WlzCRFF+S/rrITHd04iV+gDiLgw+HG?= =?Windows-1252?Q?K1Cd2Gg7m5hqkOtwtASfgsxYuIvoyrGKanEBINKYAoEjxGW5FGki+uJj?= =?Windows-1252?Q?1SaG06nKvu9S6XWbHndFu5TbB1kCd3g8yctzwOCNc6nF+o13yBTsDCHB?= =?Windows-1252?Q?VkmAzVAmn+ZRUJ1fz0sTdGpUNaCUgQt1fqg00bhpKn+ObX1zTO7mHNS6?= =?Windows-1252?Q?SoDj17Kx3sZ9bDixXztPbjPnqr34/nnQvDSgLx9S6DdOqOmjixSwdjoU?= =?Windows-1252?Q?szUJXPxfQSApHU8PXzTg0N1/StAQZJ5jF8gEBLV4qtdqxPUck97e+a+U?= =?Windows-1252?Q?wJOYFC6fl/AVYuvqeCCVNiJt1DUCP6NDOlwOb0OlQ5Rk3TLXFSw0fRZz?= =?Windows-1252?Q?DS3L4xnrjwm0hcljsoR9hrzPaB6sqFkS5HdEZOS76Z4eCcwGu9NHNCeL?= =?Windows-1252?Q?Tm6lMTcHYSFTDH2Imh4sDz82mH6sbSbM7Y+izBaQjDMOdcNuVg6GUly8?= =?Windows-1252?Q?z4lfg4d/+V4XDa6Bfr5sVBqG83u3S0qEHNR3vbJXF+RmxnMpm+BUGg6m?= =?Windows-1252?Q?lFD9gEUqSI3J915F/duBbbwN7SLEVvl4izGrvWC1cdrKDNhRvno3Pg0p?= =?Windows-1252?Q?7RRN77+FHCYS70+K60DIOElmIg7h3gJ8+ML3c/3cfV2WAU9MyWetzgsY?= =?Windows-1252?Q?U5k9hAIkDY+EQe5iESPAr+6OJcXz4i5bqaTuonh1N9GO2WFSwjSyBmUx?= =?Windows-1252?Q?GTlQRhz1LjtlYYr4F/zT818/TeJ+0dkeHXiKm3pKadkTej0zGyO/n2yW?= =?Windows-1252?Q?lhUQwzWtmYGxQz8wqrVDd4wEqovh2ExYy+zIOBhZfHN1mwWbSo5R/oQP?= =?Windows-1252?Q?ZZL1/+BWFWiMhgZUxOsvfwxL4U08TGwQC4LGgJ0aGdE=3D?= x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW3PR11MB4761.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(1800799024)(376014)(38070700018);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?Windows-1252?Q?uDWtuEfHZ4FTO6+6E7XOe5MOPJLC+7pjM1O3GB9iAOK/1M/C3QJr86f2?= =?Windows-1252?Q?pWd3x4+0KjEwmq9+ke1dgycnOaYVvUURqgrt7PKf0gnHpKkdzD/Yw02N?= =?Windows-1252?Q?Cpiqhb7vsQyzTJ1LaOzlVN80J+og+q0ulsi0SYDUOIpMBJHUnS0iwwXb?= =?Windows-1252?Q?KAGsRjrD4EHlV0XIokRzHUn74dpVpxXkHOMsP5h8bTMpB0zGLQ6lofPp?= =?Windows-1252?Q?Ve86Ilty6b90kuLfLV4mdOXAy9ejmDvfg2wF0yWRzEEEieS2Hsv/Mf6i?= =?Windows-1252?Q?DuN1nKRf77pjjeT+1OhYbC93x5KnTPz+H1w+cQUqJVml99r+hUUB4mKQ?= =?Windows-1252?Q?ps3wJDussOXl9BRsosjS5AnjbIA5be9nW5lmNR8DWIJ1k38SH/26X4Ky?= =?Windows-1252?Q?C58or++QLMmOb0JssROrX6cLD4wJTNXlxt4FiF+JrQ9+c/SyuYCPSU+n?= =?Windows-1252?Q?ibUL3NwKt2YWJhRNDtNn2QpVSYJKKeD6ur7vjCSAQhiJNd8FSePF35y9?= =?Windows-1252?Q?Ex9MKZv9xb2qgGDSsYeSEXt0STVemrQGFcG7fNIbN77PIGkiO6znzGxQ?= =?Windows-1252?Q?hyIL7I3HJIsQ7y7lyBh62NCvqSHbhfHJzFa/u+45U8Hf63wIpO7L+Qiz?= =?Windows-1252?Q?1IZrpezlk/sfJqM/x8Cmdojyg31zoTEQBIhirzqf4+4HqcmFA1oAbpkX?= =?Windows-1252?Q?qRAvg+VHAvFJI18wefocGA8piQKa4Rq7ejPrldHc8S/1nsn75uswgRkW?= =?Windows-1252?Q?iDPq/VxDDNRPpyD7KJOokobyljdvqkUXfwNMTW5gT/pcUG8m2YX/UE4b?= =?Windows-1252?Q?/bS4g/owJvQvfuXm/X+qghr2/Zr+/C2iJjpxu2IIhm8+8DPu9oWk/sMb?= =?Windows-1252?Q?I6+aadAB7sSI7q66rdYg5XcIneE/CbhUjtVNHXd0AoYRQ0Kctw/56ArG?= =?Windows-1252?Q?QQ6c18B5uGTBvut4yjKIksOa+H1Xa2xukaDgRDDhOocPtXaca3u5xaM2?= =?Windows-1252?Q?xBhqV+2S4j6rLdgknY6++rmYddiaGlXYVaDTcFe1F6KcYWxmM9u37BJB?= =?Windows-1252?Q?z8i0FWL14gYi0VIGl8BgDBz2X6KD6p1ta8BkYeRTCJkF/3QJeY1bOB8H?= =?Windows-1252?Q?MGBTfWaof/laQqkZ+OQQcuxeq30CfLkO2ODQnhxDhKFB487cXKrG7z1g?= =?Windows-1252?Q?ezBUseb8zjtzhSTvwx7uw2lhNjrH6t+LStBBUscqc/c1g5YT0in2umVP?= =?Windows-1252?Q?cBixrTWPxMzkTHGsFb8H/7iOLeH9FXOcGdE7n+Dc6xMCl1jMbqRYchei?= =?Windows-1252?Q?9ipjgypy7OkvNe/DDkyhBwUL7GPoPRYniKrjCe8XyAMOXbZLweWfoxTs?= =?Windows-1252?Q?1vgZdJAP4AX7hdeblZDRjgzYWZsVvLT1X+WRI0AljEXCnT1yP98RRRk6?= =?Windows-1252?Q?NfH/QTmI1i/nbN4ApcZoHN9jd5mycE/CiKnTfbzbh/BZZo004YZgnc7e?= =?Windows-1252?Q?h0EOODzdBh0sH/9KDHYua+qoimzc+twk6JPbIE61OhjAqzm9zuXVMvCZ?= =?Windows-1252?Q?rgcneK9Ie21nMRrezxVyukC28mqGA3I8rZummyBTEgf+3lnOrD74sYBj?= =?Windows-1252?Q?+m/bSbyLA1p4zqXfD4duG5QYTLXGHY87gs6sjF3xUZL3pRqudFlZ5ULe?= =?Windows-1252?Q?cmMBRXd/214FHSyHCQDT8HBQAMZQX4DB?= Content-Type: multipart/alternative; boundary="_000_MW3PR11MB476199009EFF9650069B752980AD2MW3PR11MB4761namp_" List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@FreeBSD.org MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MW3PR11MB4761.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: b4021be4-18c7-48f4-7715-08dca84f9603 X-MS-Exchange-CrossTenant-originalarrivaltime: 20 Jul 2024 00:04:55.2216 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 3a4zhZs8JI1EjETXWbfnWU/gwt2r2LG++K7VT0iDgya0/nxRz1+W/GZQIurosATLh+0ZnjQrRiTR0wEYB674QhFKBzefiAdItp2mvXsRd2w= X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR11MB5807 X-OriginatorOrg: intel.com X-Spamd-Bar: ---------- X-Spamd-Result: default: False [-10.67 / 15.00]; WHITELIST_SPF_DKIM(-3.00)[intel.com:d:+,intel.com:s:+]; DWL_DNSWL_MED(-2.00)[intel.com:dkim]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector10001:i=1]; NEURAL_HAM_SHORT(-0.97)[-0.975]; DMARC_POLICY_ALLOW(-0.50)[intel.com,none]; RCVD_DKIM_ARC_DNSWL_MED(-0.50)[]; R_DKIM_ALLOW(-0.20)[intel.com:s=Intel]; R_SPF_ALLOW(-0.20)[+ip4:198.175.65.0/26]; RCVD_IN_DNSWL_MED(-0.20)[198.175.65.11:from]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; FROM_HAS_DN(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_DN_SOME(0.00)[]; RCVD_TLS_LAST(0.00)[]; DKIM_TRACE(0.00)[intel.com:+]; RCVD_IN_DNSWL_NONE(0.00)[104.47.58.174:received]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_CC(0.00)[bsdimp.com,yahoo.com,freebsd.org]; MLMMJ_DEST(0.00)[freebsd-arm@freebsd.org]; MISSING_XM_UA(0.00)[]; ASN(0.00)[asn:4983, ipnet:198.175.64.0/23, country:US]; RCPT_COUNT_THREE(0.00)[4]; RCVD_COUNT_SEVEN(0.00)[7] X-Rspamd-Queue-Id: 4WQmy667Z2z4h0s --_000_MW3PR11MB476199009EFF9650069B752980AD2MW3PR11MB4761namp_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable Hi, The location from which we are trying to boot FreeBSD is a VFS or RAMDisk m= ethod: OK lsdev disk devices: disk0: 4194304 X 512 blocks disk0p1: EFI disk0p2: FreeBSD UFS http: (unknown) net devices: net0: net1: net2: net3: Looks like FreeBSD does not have access to this RAMDisk and hence unable to= boot the image. I was digging through FreeBSD forums and found some releva= nt details. I guess the first step would be to build FreeBSD to prepare it = as RAMDisk image, so from the links below i could gather something like thi= s ? Some Examples: https://forums.freebsd.org/threads/how-would-i-load-root-image-into-memory-= backed-fs-and-run-the-system-from-that.78021/ https://www.reddit.com/r/freebsd/comments/tzm2xi/run_from_ram/ https://forums.freebsd.org/threads/mount-memory-disk-on-boot.61090/ https://forums.freebsd.org/threads/questions-about-ramdisk-in-freebsd.20345= / mdmfs (https://man.freebsd.org/cgi/man.cgi?query=3Dmdmfs&sektion=3D8&manpat= h=3Dfreebsd-release-ports#EXAMPLES) and mdconfig(https://man.freebsd.org/cg= i/man.cgi?query=3Dmdconfig&sektion=3D8&manpath=3Dfreebsd-release-ports): 1. Prepare RAMDisk image mdconfig -a -t vnode -s 10g -u 0 newfs /dev/md0 mount /dev/md0 /mnt dump -0aLf - / | (cd /mnt && restore -rf -) dd if=3D/dev/md0 of=3D/path/to/ramdisk.img bs=3D1M 2. Edit /boot/loader.conf: initrd_load=3D"YES" initrd_type=3D"mfs_root" initrd_name=3D"/boot/ramdisk.img" vfs.root.mountfrom=3D"ufs:/dev/md0" vfs.root.mountfrom.options=3D"ro,noatime" 3. Set Boot Variables in the Bootloader OK set initrd_load=3D"YES" OK set initrd_type=3D"mfs_root" OK set initrd_name=3D"/boot/ramdisk.img" OK set vfs.root.mountfrom=3D"ufs:/dev/md0" OK set vfs.root.mountfrom.options=3D"ro,noatime" OK boot Can someone please clarify if this is a right approach? Any inputs are appr= eciated!! Thanks, Arjun ________________________________ From: Andrew Turner Sent: Wednesday, July 10, 2024 11:36 AM To: Anantharam, Arjun Cc: Warner Losh ; Mark Millard ; freebsd= -arm@freebsd.org Subject: Re: FreeBSD OS installation stuck after selecting Boot installer On 10 Jul 2024, at 17:03, Anantharam, Arjun wr= ote: Hi, On most of our hardware, i still run into mount issues. Can anyone provide = some guidance on this ? This iso/img file boot is the only option right now for us to boot FreeBSD(= no PXE boot option yet), so any inputs are appreciated!! The issue appears to be the interface the NVMe provides to the kernel. found-> vendor=3D0x8086, dev=3D0x1457, revid=3D0x21 domain=3D0, bus=3D0, slot=3D1, func=3D1 class=3D01-08-02, hdrtype=3D0x00, mfdev=3D1 cmdreg=3D0x0000, statreg=3D0x0010, cachelnsz=3D0 (dwords) lattimer=3D0x00 (0 ns), mingnt=3D0x00 (0 ns), maxlat=3D0x00 (0 ns) intpin=3Da, irq=3D255 powerspec 3 supports D0 D3 current D0 MSI supports 1 message, 64 bit, vector masks MSI-X supports 257 messages in map 0x18 map[10]: type Prefetchable Memory, range 64, base 0x2220000000, siz= e 25, memory disabled This looks like the NVMe device that the nvme driver attaches to. The class= =3D01-08-02 indicates it=92s an NVMe device using the enterprise nvmhci. =85 found-> vendor=3D0x8086, dev=3D0x1458, revid=3D0x21 domain=3D0, bus=3D0, slot=3D1, func=3D7 class=3D01-08-00, hdrtype=3D0x00, mfdev=3D1 cmdreg=3D0x0000, statreg=3D0x0010, cachelnsz=3D0 (dwords) lattimer=3D0x00 (0 ns), mingnt=3D0x00 (0 ns), maxlat=3D0x00 (0 ns) intpin=3Da, irq=3D255 powerspec 3 supports D0 D3 current D0 MSI-X supports 257 messages in map 0x18 map[10]: type Prefetchable Memory, range 64, base 0x2224000000, siz= e 25, memory disabled This is the NVMe device the nvme driver doesn=92t attach to. As the class i= s 01-08-00 the driver doesn=92t attach. The problem is the 00 at the end. I don=92t know the difference between the different interfaces so am unsure= how difficult it would be to support this device. Andrew --_000_MW3PR11MB476199009EFF9650069B752980AD2MW3PR11MB4761namp_ Content-Type: text/html; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable
Hi,

The location from which we are trying to boot FreeBSD is a VFS or RAMDisk m= ethod:
OK lsdev
disk devices:
     disk0:    4194304 X 512 blocks
       disk0p1: EFI
      disk0p2: FreeBSD UFS
 http: (unknown)
 net devices:
    net0:
     net1:
     net2:
     net3:


Looks like FreeBSD does not have access to this RAMDisk and hence unable to= boot the image. I was digging through FreeBSD forums and found some releva= nt details. I guess the first step would be to build FreeBSD to prepare it = as RAMDisk image, so from the links below i could gather something like this ?
Some Examples:
https://www.reddit.com/r/freebsd/comments/tzm2xi/run_from_ram/
 
mdmfs (https://man.freebsd.org/cgi/man.cgi?query= =3Dmdmfs&sektion=3D8&manpath=3Dfreebsd-release-ports#EXAMPLES) and mdconfig(https://man.freebsd.org/cgi/man.cgi?query=3Dmdc= onfig&sektion=3D8&manpath=3Dfreebsd-release-ports):
  1. Prepare RAMDisk image
mdconfig -a -t vnode -s 10g -u 0
newfs /dev/md0
mount /dev/md0 /mnt
dump -0aLf - / | (cd /mnt && restore -rf -)
dd if=3D/dev/md0 of=3D/path/to/ramdisk.img bs=3D1M
      2. Edit /boot/loader.conf:
 
initrd_load=3D"YES"
initrd_type=3D"mfs_root"
initrd_name=3D"/boot/ramdisk.img"
vfs.root.mountfrom=3D"ufs:/dev/md0"
vfs.root.mountfrom.options=3D"ro,noatime"
 
    3. Set Boot Variables in the Bootloader
 
OK set initrd_load=3D"YES"
OK set initrd_type=3D"mfs_root"
OK set initrd_name=3D"/boot/ramdisk.img"
OK set vfs.root.mountfrom=3D"ufs:/dev/md0"
OK set vfs.root.mountfrom.options=3D"ro,noatime"
OK boot

Can someone please clarify if this is a right approach? Any inputs are appr= eciated!!

Thanks,
Arjun
Hi,

On most of our hardware, i still run into mount issues. Can anyone provide = some guidance on this ?
This iso/img file boot is the only option right now for us to boot FreeBSD(= no PXE boot option yet), so any inputs are appreciated!!


The issue appears to be the interface the NVMe provides to the kernel.=

found-> vendor=3D0x8086, dev=3D0x1457, revid=3D0x21
        domain=3D0, bus=3D0, slot=3D1, func=3D1
        class=3D01-08-02, hdrtype=3D0x00, mfdev=3D1
        cmdreg=3D0x0000, statreg=3D0x0010, cachelnsz=3D= 0 (dwords)
        lattimer=3D0x00 (0 ns), mingnt=3D0x00 (0 ns), m= axlat=3D0x00 (0 ns)
        intpin=3Da, irq=3D255
        powerspec 3  supports D0 D3  current = D0
        MSI supports 1 message, 64 bit, vector masks
        MSI-X supports 257 messages in map 0x18
        map[10]: type Prefetchable Memory, range 64, ba= se 0x2220000000, size 25, memory disabled

This looks like the NVMe device that the nvme driver attaches to. The&= nbsp;class=3D01-08-02 indicates it=92s an NVMe device using the enterprise = nvmhci.

=85

found-> vendor=3D0x8086, dev=3D0x1458, revid=3D0x21
        domain=3D0, bus=3D0, slot=3D1, func=3D7
        class=3D01-08-00, hdrtype=3D0x00, mfdev=3D1
        cmdreg=3D0x0000, statreg=3D0x0010, cachelnsz=3D= 0 (dwords)
        lattimer=3D0x00 (0 ns), mingnt=3D0x00 (0 ns), m= axlat=3D0x00 (0 ns)
        intpin=3Da, irq=3D255
        powerspec 3  supports D0 D3  current = D0
        MSI-X supports 257 messages in map 0x18
        map[10]: type Prefetchable Memory, range 64, ba= se 0x2224000000, size 25, memory disabled

This is the NVMe device the nvme driver doesn=92t attach to. As the cl= ass is 01-08-00 the driver doesn=92t attach. The problem is the 00 at the e= nd.

I don=92t know the difference between the different interfaces so am u= nsure how difficult it would be to support this device.

Andrew
--_000_MW3PR11MB476199009EFF9650069B752980AD2MW3PR11MB4761namp_-- From nobody Sat Jul 20 05:38:36 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WQwMM0ZzQz5QyJW for ; Sat, 20 Jul 2024 05:38:55 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic316-8.consmr.mail.gq1.yahoo.com (sonic316-8.consmr.mail.gq1.yahoo.com [98.137.69.32]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4WQwML2z1Dz4LQd for ; Sat, 20 Jul 2024 05:38:54 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=VfdEiul5; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.69.32 as permitted sender) smtp.mailfrom=marklmi@yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1721453931; bh=UQpBBy/tN7gmY+ydAyAs3gPyeju1tGEkIkY/m20z7lQ=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=VfdEiul5qeFI1uOKuhF0ekLeqqWvyVcr2GTkZD1ubKC/xfQo1HIKbk5UieZ04pxvM9hEijjuB/pmt+tKgCqOeX+6ugY5aMMDCKwJ/Sz3fHbTGUVdeMrajb4TzIgmZf+99ctW0702GGuPJAX1wCHlCEkglQYdxKSjs8/iiM2CS+p9RZhn48eW7GN4OZTbtphdB0NodOSXoEZBCXJHWK5XI6AO9Xqozpc9mhXgsovtDLxgTkPhFMcMLWyopVTlBvxYmNIHe1gHLPDq0MREEPh/CblaKtL9XHMmHLZJZdXopS1r/ZRRK0xJekrSeljOOQX5JvJoU+5Z0sMqChgyW/7sLQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1721453931; bh=7mdU6X/G8vu0+UAFVTchC9+tJgcZNGadJFoee0kS6M8=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=H410I1d04nuI5YrXFHN7rtwih10r3gl89esT3MhWSPF/WPVsGJFjGtC1QUiCgWxEiTn03FAvTpXzVD+/LSiSvnGvL0v+JrqNU+KWX8l2/UEMLhQQVJRe3gbmpV80fjkHPMfOmOKCNIqBKaYQMlqZID+OlA03B3/VJnXTpjQGeGr554ytRSaUqXUU/sDBHfqZqulNeNS1i8FVo8XAy/KGNh8GuiLlBcE1y5NEw4SoOfZE2j1qum4WLP3lGEnIbC5mDNlV7m04IEJUs+RiamnqsClO4fvhQ5Qbq31qvN3/btj2ag/LrZEuBo2WMqTjFVYIWi4nz12RxKgBIKFynjC/CQ== X-YMail-OSG: .vTUtb4VM1kfCJLJSwIJLuOA5InlpId7VvOXWpj_WzWqGbflC7Itbbad1qeumXv MrAZgRXwlhtSyw7AJMnzHbdBj1_xTdSGyeaWKpVhsY1ORIv3YgPx.W0ktlnPkAMcuLYkREbtmGlS 0Zqu2y8BqeU9x7ScsReapFJMXhYjoUIggHFGFzL0kPJn9OdR1Yt9eAuME9icFkDRlVvEjHavLMzC 4SWXXzmi8AAxLD0l7K.rKo928SWpEY9Lx83Qh9aMdFXjmQL.avOoKt7VlnG9xASDJFpV3jGRhfhD QdOc4LEct9qzAKnB5kaiVO_jdF8y7OY8XDFQj17maqfmds4lmkMp5AlmyjfXuUP_nx8FutpAcoqs 4lQOp0DYfxaj7LU3FthW7c.Cqwjv.rbLxcHmLImFfknq8Owb9WFgt6AItuNM3aFhJO7.qWycEXaq I5WjA7DBzad4HWegXyXVilBzml7hsa1uazFUUsTO7hpt6r7s12c7K7Qml8fGjYtizPsVT_V5rNBz i4KR0sxY0Le1B5UmuyT2ctXx1zzwo6LxMmvEPSO4lGDN9Q3xKaXPHsBeUeehc2g71QO12RBzdPyV 2F06QEiEP6WhF1kjPIpRqmsUI0j_udbVOpvl.GvgfyHaBG7JYfDnsO1L4gHLJXCEf9.7ODqzixnb dR0QazOpD7aZFm8WMZg3Efh6cWgvSe35w6mO5Q.lIY6dv2pIzRt.PFEUxVJg00.J0a.5ad7dCgQR DnvlS4Ocy5tZE1gxQ0FOpFB4JN1KUADM1b3rya3D7etEY294sqIjbqmduXrck13A01Fc_Kan4qbL pw9MuWHsJoQk0AljVkzk6TtgpNRgrBxc0DAtSooUCRzSHNdAaco5ysxQqESiQYSP8W4sanxIcywW l7I8gzKvGuNxvwfQfl.uZoR7FMCOtOL99Xni4v8bRm6.D9bNpWCQPD96oQHJaTBx3iwndIFoeYWB irL_i0._UO_h.QGAMjXJeZh41XTKIxnsmbbtQmOaLJTTRMOuxiXhlSTg0nXgoVKoV0lOxx3Sfefp .dvFcaal5fE3AaP3LEjFWPd_lrRX2QOZH3YiGWW58gnh6LM2_ZQoz.hN7mKPpOVL.asYPM.3Kjri 85Hb3tQqhHNgkI4N65MKEfUiWizp_CpVw3nuNOClYByagBbiT6bFhMxkbUm.ck3vx8b0OkuUpZIt RUBNcIkQcUvPlYeEnJSFuK6vhQwLw.kcwTMKVKK9g18.oeOmC25M.0gd3KXn1yrAtp_UgiF2Mojx 61cUwi9p4JXtfafdfKwoZU68FaT906MsKM02fl4C.nf4aSp.3b.9SySThoXQTKkiFC35Qgv8QIOi v5D70A3gHTnxyL4SRZ2zk0x8ojwFzkGkhffkWp_d5z9PTUw8hddXgA_8mSxHX4ezVuQXH5d7GuiT 47YE.lSzhj70hhQIVwY0fq.86JXbeZnmzQqh9.SChuoXEPRa72u91JcZthV7wqzHkYoemIVUWu5x qZdLiTAOOtkoThEY9CJM5onXLNJqT3.bh7EtfywVlSnkvWY16Iyltyjwck1dXzByWwmMuv9lCIpt GGomSUIqYaYR36oTJ7yJg_Phv2ciKx1BNQF2RBuUrlD76Cxe00ArrEIYVW8Yz05_VdsndIu2Es0v fSHoL_uQvM9VDoAvjPAZumMXAnqln7uGkVbYXU6p_fJZUgqcIbClC19D90qBbPVRaGHFV52t_usq VA0kt2h4bWidCBpKncB0rQ1La30RRFhzpGQUEj4NxpvWjiU4M17Wt49.KSWJHp52m3GHSnU3Bch_ 46wO1TeMcqJwpPnks6gBHdV.yzmLNHKl.8mS1FlmTlC8.b3W04GCFyt8XzLgL_AXqXrYzz6kn3cq R9rN0b9H9QNIgD_mAiwCF03x6MzsfP3VGJNTQJey28xluPCKj.ewochJYEb_Bhf9obXVRSnBK3sI NQw2xDHaNna9OFsSYsfA8YLILYZIDGuDusqpVjBYuy8_AtfgR8cvuGjvLhV.eT1XWoTN9THDkzgY fN2ZE2HUXjJtnIG2xpU29gNV4X0zkrcnnCoJJAF9nPrYajZgy3o0fjiZPlxiubJOs0E61TypASvn 6MCpOuvUk3LFsfQAfnUfMx0GbzbzrUkB1Wt9PAmQ03CPMANIytemeaBXf8oPWAI6Jvk8PRJkgKXb JLVKqJaMz7TcLseIO37J9qk3iyWbVLLsq4uao_ngTU1e8teXa2ahtHDNNQvyD27IeErvzjLnrZx0 iVW.DE0km.9D56bfOluT.OzVA42dxxEQR3YZrfiSrZJUVNrQaqeKhUDCE5z_iywwDPpbJCic_AtD w1dogUuA- X-Sonic-MF: X-Sonic-ID: 4126e849-f064-4f2d-9c0f-863a11c73ae1 Received: from sonic.gate.mail.ne1.yahoo.com by sonic316.consmr.mail.gq1.yahoo.com with HTTP; Sat, 20 Jul 2024 05:38:51 +0000 Received: by hermes--production-gq1-799bb7c8cf-b6h6x (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 57cb430acfaaab9128e7287d803f76ba; Sat, 20 Jul 2024 05:38:47 +0000 (UTC) Content-Type: text/plain; charset=utf-8 List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.600.62\)) Subject: Re: armv7-on-aarch64 stuck at urdlck: I got a replication of the "ampere2" bulk build hangup problem on a Windows DevKit 2023 From: Mark Millard In-Reply-To: Date: Fri, 19 Jul 2024 22:38:36 -0700 Cc: Konstantin Belousov , Baptiste Daroussin Content-Transfer-Encoding: quoted-printable Message-Id: References: <8214703E-AB28-4FB3-A3DD-03C87363D8C6@yahoo.com> To: "olce@freebsd.org" , Philip Paeps , FreeBSD ARM List , FreeBSD Mailing List , Current FreeBSD X-Mailer: Apple Mail (2.3774.600.62) X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.98 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.98)[-0.982]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; DKIM_TRACE(0.00)[yahoo.com:+]; FREEMAIL_FROM(0.00)[yahoo.com]; TO_DN_SOME(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MLMMJ_DEST(0.00)[freebsd-arm@freebsd.org]; RCPT_COUNT_SEVEN(0.00)[7]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.32:from]; APPLE_MAILER_COMMON(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.32:from] X-Rspamd-Queue-Id: 4WQwML2z1Dz4LQd On Jul 18, 2024, at 01:14, Mark Millard wrote: > On Jul 16, 2024, at 23:45, Mark Millard wrote: >=20 >> On Jul 16, 2024, at 18:41, Mark Millard wrote: >>=20 >>> On Jul 16, 2024, at 11:37, Mark Millard wrote: >>>=20 >>>> On Jul 16, 2024, at 10:42, Mark Millard wrote: >>>>=20 >>>>> No longer is the problem only observed on ampere2! But this was = with >>>>> a non-debug, personally built kernel that has some of my now = patches. >>>>> I'll see if I can replicate the issue with an official pkgbase = debug >>>>> kernel. >>>>=20 >>>> It replicated with the official pkgbase debug kernel. The >>>> kernel did not report anything. >>>>=20 >>>> The following commits in main and happen between the last working >>>> ampere2 armv7 builds and the first failing ampere2 builds and >>>> look be the only likely contributors from that range as far as >>>> I could tell: >>>>=20 >>>> Tue, 27 Feb 2024 >>>> . . . >>>> =E2=80=A2 git: 1df8700aa6cf - main - PP mutexes: unlock: Reset = inherited prio regardless of privileges Olivier Certner=20 >>>> =E2=80=A2 git: 9ac3ac9ece62 - main - PP mutexes: lock: Check if = priority is too high against base one Olivier Certner=20 >>>> =E2=80=A2 git: 39e4665c9694 - main - PP mutexes: lock: Reduce = 'umtx_lock' holding before taking the user lock Olivier Certner >>>>=20 >>>> These changes are not in 14.0-RELEASE but are in 14.1-STABLE and = 14.1-RELEASE. >>>> So I expect that when any ampere*'s progress to 14.1-RELEASE the = armv7 >>>> problems would start for them. >>>>=20 >>>> These changes are not in 13.3-RELEASE but are in 13.3-STABLE. So I = expect that >>>> when any ampere*'s progress to 13.4-RELEASE the problems would = start for them. >>>>=20 >>>>=20 >>>> With the prior packages already built in a prior poudriere-devel >>>> run it turns out that just: >>>>=20 >>>> # poudriere bulk -j main-armv7-poud -i graphics/graphviz >>>>=20 >>>> replicates the problem: >>>>=20 >>>> . . . >>>> [00:00:45] Installing graphics/graphviz | graphviz-9.0.0_4 >>>> [aarch64PBase] Installing graphviz-9.0.0_4... >>>> [aarch64PBase] `-- Installing cairo-1.17.4_2,3... >>>> [aarch64PBase] | `-- Installing fontconfig-2.15.0_2,1... >>>> [aarch64PBase] | | `-- Installing expat-2.6.2... >>>> [aarch64PBase] | | `-- Extracting expat-2.6.2: 100% >>>> [aarch64PBase] | | `-- Installing freetype2-2.13.2... >>>> [aarch64PBase] | | `-- Installing brotli-1.1.0,1... >>>> [aarch64PBase] | | `-- Extracting brotli-1.1.0,1: 100% >>>> [aarch64PBase] | | `-- Installing png-1.6.43... >>>> [aarch64PBase] | | `-- Extracting png-1.6.43: 100% >>>> [aarch64PBase] | | `-- Extracting freetype2-2.13.2: 100% >>>> [aarch64PBase] | `-- Extracting fontconfig-2.15.0_2,1: 100% >>>> [aarch64PBase] | `-- Installing glib-2.80.4,2... >>>> [aarch64PBase] | | `-- Installing libffi-3.4.6... >>>> [aarch64PBase] | | `-- Extracting libffi-3.4.6: 100% >>>> [aarch64PBase] | | `-- Installing libiconv-1.17_1... >>>> [aarch64PBase] | | `-- Extracting libiconv-1.17_1: 100% >>>> [aarch64PBase] | | `-- Installing pcre2-10.43... >>>> [aarch64PBase] | | `-- Extracting pcre2-10.43: 100% >>>> [aarch64PBase] | | `-- Installing py311-packaging-24.1... >>>> [aarch64PBase] | | `-- Installing python311-3.11.9_1... >>>> [aarch64PBase] | | | `-- Installing mpdecimal-4.0.0... >>>> [aarch64PBase] | | | `-- Extracting mpdecimal-4.0.0: 100% >>>> [aarch64PBase] | | | `-- Installing readline-8.2.10... >>>> [aarch64PBase] | | | `-- Extracting readline-8.2.10: 100% >>>> [aarch64PBase] | | `-- Extracting python311-3.11.9_1: 100% >>>> [aarch64PBase] | | `-- Extracting py311-packaging-24.1: 100% >>>> [aarch64PBase] | `-- Extracting glib-2.80.4,2: 100% >>>> [aarch64PBase] | `-- Installing libglvnd-1.7.0... >>>> [aarch64PBase] | `-- Extracting libglvnd-1.7.0: 100% >>>> [aarch64PBase] | `-- Installing pixman-0.42.2... >>>> [aarch64PBase] | `-- Extracting pixman-0.42.2: 100% >>>> [aarch64PBase] `-- Extracting cairo-1.17.4_2,3: 100% >>>> [aarch64PBase] `-- Installing harfbuzz-9.0.0... >>>> [aarch64PBase] | `-- Installing graphite2-1.3.14... >>>> [aarch64PBase] | `-- Extracting graphite2-1.3.14: 100% >>>> [aarch64PBase] `-- Extracting harfbuzz-9.0.0: 100% >>>> [aarch64PBase] `-- Installing jpeg-turbo-3.0.3... >>>> [aarch64PBase] `-- Extracting jpeg-turbo-3.0.3: 100% >>>> [aarch64PBase] `-- Installing libgd-2.3.3_13,1... >>>> [aarch64PBase] | `-- Installing tiff-4.6.0... >>>> [aarch64PBase] | | `-- Installing jbigkit-2.1_3... >>>> [aarch64PBase] | | `-- Extracting jbigkit-2.1_3: 100% >>>> [aarch64PBase] | | `-- Installing lerc-4.0.0... >>>> [aarch64PBase] | | `-- Extracting lerc-4.0.0: 100% >>>> [aarch64PBase] | | `-- Installing libdeflate-1.20... >>>> [aarch64PBase] | | `-- Extracting libdeflate-1.20: 100% >>>> [aarch64PBase] | | `-- Installing zstd-1.5.6... >>>> [aarch64PBase] | | `-- Installing liblz4-1.9.4_1,1... >>>> [aarch64PBase] | | `-- Extracting liblz4-1.9.4_1,1: 100% >>>> [aarch64PBase] | | `-- Extracting zstd-1.5.6: 100% >>>> [aarch64PBase] | `-- Extracting tiff-4.6.0: 100% >>>> [aarch64PBase] | `-- Installing webp-1.4.0_1... >>>> [aarch64PBase] | | `-- Installing giflib-5.2.2... >>>> [aarch64PBase] | | `-- Extracting giflib-5.2.2: 100% >>>> [aarch64PBase] | `-- Extracting webp-1.4.0_1: 100% >>>> [aarch64PBase] `-- Extracting libgd-2.3.3_13,1: 100% >>>> [aarch64PBase] `-- Installing libltdl-2.4.7... >>>> [aarch64PBase] `-- Extracting libltdl-2.4.7: 100% >>>> [aarch64PBase] `-- Installing pango-1.52.2_1... >>>> [aarch64PBase] | `-- Installing fribidi-1.0.15... >>>> [aarch64PBase] | `-- Extracting fribidi-1.0.15: 100% >>>> [aarch64PBase] | `-- Installing libXft-2.3.8... >>>> [aarch64PBase] | `-- Extracting libXft-2.3.8: 100% >>>> [aarch64PBase] | `-- Installing libthai-0.1.29_1... >>>> [aarch64PBase] | | `-- Installing libdatrie-0.2.13_2... >>>> [aarch64PBase] | | `-- Extracting libdatrie-0.2.13_2: 100% >>>> [aarch64PBase] | `-- Extracting libthai-0.1.29_1: 100% >>>> [aarch64PBase] `-- Extracting pango-1.52.2_1: 100% >>>> [aarch64PBase] Extracting graphviz-9.0.0_4: 100% >>>>=20 >>>> And here it is hung with /usr/local/bin/dot -c in urdlck : >>>>=20 >>>> 0 1483 4502 7 68 0 15760 4872 wait I+ 0 0:03.92 | | = `-- /usr/local/libexec/poudriere/sh -e -o pipefail = /usr/local/share/poudriere/bulk.sh -j main-armv7-poud -i = graphics/graphviz >>>> 0 1894 1483 5 68 0 15760 4712 nanslp S 0 0:02.07 | | = |-- sh: poudriere[main-armv7-poud-default]: html_json_main (sh) >>>> 0 25321 1483 6 68 0 6664 3868 wait I+J 0 0:00.11 | | = `-- /usr/bin/make -C /usr/ports/graphics/graphviz install-package >>>> 0 25322 25321 5 68 0 11140 8860 wait I+J 0 0:00.00 | | = `-- /usr/local/sbin/pkg-static add = /packages/All/graphviz-9.0.0_4.pkg >>>> 0 25323 25322 5 20 0 63824 45144 select S+J 0 0:02.85 | | = `-- /usr/local/sbin/pkg-static add = /packages/All/graphviz-9.0.0_4.pkg >>>> 0 26900 25323 3 68 0 26292 23804 urdlck I+J 0 0:00.02 | | = `-- /usr/local/bin/dot -c >>>>=20 >>>>=20 >>>>> FYI for the replication that I got: >>>>>=20 >>>>> /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>>>> runs: >>>>> /usr/local/bin/dot -c >>>>>=20 >>>>> each such /usr/local/bin/dot is stuck at MWCHAN urdlck . >>>>>=20 >>>>> # poudriere status -b >>>>> [main-armv7-poud-default] [2024-07-16_04h27m31s] [parallel_build] = Queued: 449 Built: 433 Failed: 0 Skipped: 0 Ignored: 0 Fetched: 0 = Tobuild: 16 Time: 04:44:38 >>>>> ID TOTAL ORIGIN PKGNAME = PHASE PHASE TMPFS CPU% MEM% >>>>> [01] 00:59:45 graphics/rubygem-ruby-graphviz | = rubygem-ruby-graphviz-1.2.5 run-depends 00:59:30 1.59 GiB 0% 0.2% >>>>> [02] 00:49:59 graphics/p5-GraphViz | p5-GraphViz-2.25 = build-depends 00:49:53 1.45 GiB 0% 0.2% >>>>> [03] 00:59:45 graphics/py-pydot@py311 | py311-pydot-2.0.0 = run-depends 00:59:34 1.47 GiB 0% 0.2% >>>>> [04] 00:59:45 graphics/py-pygraphviz@py311 | = py311-pygraphviz-1.6 lib-depends 00:59:33 1.47 GiB 0% 0.2% >>>>> [05] 00:58:57 graphics/py-graphviz@py311 | = py311-graphviz-0.10.1 run-depends 00:58:49 1.47 GiB 0% 0.2% >>>>> [06] 00:59:20 audio/ganv | ganv-1.8.2_1 = lib-depends 00:59:10 1.53 GiB 0% 0.2% >>>>> [07] 00:59:45 devel/libr3 | libr3-1.0.0_2 = lib-depends 00:59:19 1.54 GiB 0% 0.3% >>>>> [08] 00:59:45 net/netmap | netmap-0.1.3_2 = run-depends 00:59:22 1.46 GiB 0% 0.3% >>>>>=20 >>>>> I had started the bulk build via the list: >>>>>=20 >>>>> # more ~/origins/ampere2-failures-armv7.txt >>>>> audio/ganv >>>>> devel/doxygen >>>>> devel/libr3 >>>>> graphics/p5-GraphViz >>>>> graphics/p5-GraphViz2 >>>>> graphics/oyranos >>>>> graphics/pear-Image_GraphViz@php81 >>>>> graphics/py-graphviz@py311 >>>>> graphics/py-pydot@py311 >>>>> graphics/py-pygraphviz@py311 >>>>> graphics/rubygem-ruby-graphviz >>>>> math/ggobi >>>>> net-mgmt/librenms >>>>> net/netmap >>>>> print/dot2tex@py311 >>>>>=20 >>>>> # poudriere bulk -j main-armv7-poud `cat = ~/origins/ampere2-failures-armv7.txt` >>>>> . . . >>>>> [00:00:12] Building 449 packages using up to 8 builders >>>>> . . . >>>>> [03:44:55] [01] [00:18:54] Finished graphics/graphviz | = graphviz-9.0.0_4: Success >>>>> [03:44:56] [01] [00:00:00] Building = graphics/rubygem-ruby-graphviz | rubygem-ruby-graphviz-1.2.5 >>>>> [03:44:56] [03] [00:00:00] Building graphics/py-pydot@py311 | = py311-pydot-2.0.0 >>>>> [03:44:56] [04] [00:00:00] Building graphics/py-pygraphviz@py311 = | py311-pygraphviz-1.6 >>>>> [03:44:56] [07] [00:00:00] Building devel/libr3 | libr3-1.0.0_2 >>>>> [03:44:56] [08] [00:00:00] Building net/netmap | netmap-0.1.3_2 >>>>> [03:45:21] [06] [00:09:33] Finished x11-toolkits/gtkmm24 | = gtkmm24-2.24.5_4: Success >>>>> [03:45:21] [06] [00:00:00] Building audio/ganv | ganv-1.8.2_1 >>>>> [03:45:44] [05] [00:13:33] Finished graphics/ImageMagick6@nox11 = | ImageMagick6-nox11-6.9.12.77_9,1: Success >>>>> [03:45:44] [05] [00:00:00] Building graphics/py-graphviz@py311 | = py311-graphviz-0.10.1 >>>>> [03:54:42] [02] [00:24:53] Finished print/texlive-base | = texlive-base-20240312: Success >>>>> [03:54:42] [02] [00:00:00] Building graphics/p5-GraphViz | = p5-GraphViz-2.25 >>>>>=20 >>>>> In /usr/src/sys/kern/kern_umtx.c there is: >>>>>=20 >>>>> static int >>>>> do_rw_rdlock(struct thread *td, struct urwlock *rwlock, long = fflag, >>>>> struct _umtx_time *timeout) >>>>> { >>>>> . . . >>>>> /* >>>>> * Contention bit is set, before sleeping, increase >>>>> * read waiter count. >>>>> */ >>>>> rv =3D fueword32(&rwlock->rw_blocked_readers, >>>>> &blocked_readers); >>>>> if (rv =3D=3D 0) >>>>> rv =3D suword32(&rwlock->rw_blocked_readers, >>>>> blocked_readers + 1); >>>>> if (rv =3D=3D -1) { >>>>> umtxq_unbusy_unlocked(&uq->uq_key); >>>>> error =3D EFAULT; >>>>> break; >>>>> } >>>>> while (state & wrflags) { >>>>> umtxq_lock(&uq->uq_key); >>>>> umtxq_insert(uq); >>>>> umtxq_unbusy(&uq->uq_key); >>>>> error =3D umtxq_sleep(uq, "urdlck", timeout = =3D=3D NULL ? >>>>> NULL : &timo); >>>>> umtxq_busy(&uq->uq_key); >>>>> umtxq_remove(uq); >>>>> umtxq_unlock(&uq->uq_key); >>>>> if (error) >>>>> break; >>>>> rv =3D fueword32(&rwlock->rw_state, &state); >>>>> if (rv =3D=3D -1) { >>>>> error =3D EFAULT; >>>>> break; >>>>> } >>>>> } >>>>>=20 >>>>> . . . >>>>>=20 >>>>>=20 >>>>>=20 >>>>> For reference: >>>>>=20 >>>>> # ps -alxdww | less >>>>> UID PID PPID C PRI NI VSZ RSS MWCHAN STAT TT TIME = COMMAND >>>>> . . . >>>>> 0 87700 4522 6 20 0 16576 1888 - T 0 0:00.01 | = | |-- vi /usr/local/share/poudriere/jail.sh >>>>> 0 91496 4522 4 20 0 15760 4684 select S+ 0 0:06.88 | = | `-- /usr/local/libexec/poudriere/sh -e -o pipefail = /usr/local/share/poudriere/bulk.sh audio/ganv devel/doxygen devel/libr3 = graphics/p5-GraphViz graphics/p5-GraphViz2 graphics/oyranos = graphics/pear-Image_GraphViz@php81 graphics/py-graphviz@py311 = graphics/py-pydot@py311 graphics/py-pygraphviz@py311 = graphics/rubygem-ruby-graphviz math/ggobi net-mgmt/librenms net/netmap = print/dot2tex@py311 >>>>> 0 37688 91496 0 68 0 15760 4700 wait I 0 0:00.05 | = | |-- sh: poudriere[main-armv7-poud-default][01]: build_pkg = (rubygem-ruby-graphviz-1.2.5) (sh) >>>>> 0 47568 37688 0 68 0 6664 3664 wait IJ 0 0:00.03 | = | | `-- /usr/bin/make -C /usr/ports/graphics/rubygem-ruby-graphviz = run-depends >>>>> 0 47598 47568 6 68 0 5568 2988 wait IJ 0 0:00.01 | = | | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh >>>>> 0 47743 47598 6 68 0 11188 8864 wait IJ 0 0:00.00 | = | | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>>>> 0 47747 47743 3 20 0 71692 48984 select SJ 0 0:04.26 | = | | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>>>> 0 56383 47747 1 68 0 26292 23812 urdlck IJ 0 0:00.02 | = | | `-- /usr/local/bin/dot -c >>>>> 0 37700 91496 6 68 0 15760 4700 wait I 0 0:00.04 | = | |-- sh: poudriere[main-armv7-poud-default][03]: build_pkg = (py311-pydot-2.0.0) (sh) >>>>> 0 45102 37700 2 68 0 6668 3704 wait IJ 0 0:00.02 | = | | `-- /usr/bin/make -C /usr/ports/graphics/py-pydot FLAVOR=3Dpy311 = run-depends >>>>> 0 45156 45102 4 68 0 5584 2992 wait IJ 0 0:00.01 | = | | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh >>>>> 0 45215 45156 4 68 0 11144 8864 wait IJ 0 0:00.00 | = | | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>>>> 0 45218 45215 4 20 0 51420 31512 select SJ 0 0:02.68 | = | | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>>>> 0 52147 45218 2 68 0 26292 23812 urdlck IJ 0 0:00.02 | = | | `-- /usr/local/bin/dot -c >>>>> 0 37721 91496 1 68 0 15760 4700 wait I 0 0:00.04 | = | |-- sh: poudriere[main-armv7-poud-default][04]: build_pkg = (py311-pygraphviz-1.6) (sh) >>>>> 0 45937 37721 1 68 0 6684 3744 wait IJ 0 0:00.03 | = | | `-- /usr/bin/make -C /usr/ports/graphics/py-pygraphviz = FLAVOR=3Dpy311 lib-depends >>>>> 0 46009 45937 7 68 0 5584 2992 wait IJ 0 0:00.01 | = | | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh >>>>> 0 46127 46009 7 68 0 11144 8864 wait IJ 0 0:00.00 | = | | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>>>> 0 46129 46127 1 20 0 51384 31548 select SJ 0 0:02.73 | = | | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>>>> 0 53311 46129 4 68 0 26292 23812 urdlck IJ 0 0:00.02 | = | | `-- /usr/local/bin/dot -c >>>>> 0 37744 91496 7 45 0 15760 4692 wait I 0 0:00.04 | = | |-- sh: poudriere[main-armv7-poud-default][07]: build_pkg = (libr3-1.0.0_2) (sh) >>>>> 0 55198 37744 0 50 0 6664 3664 wait IJ 0 0:00.03 | = | | `-- /usr/bin/make -C /usr/ports/devel/libr3 lib-depends >>>>> 0 55229 55198 0 68 0 5588 2988 wait IJ 0 0:00.01 | = | | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh >>>>> 0 55594 55229 7 68 0 11168 8864 wait IJ 0 0:00.00 | = | | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>>>> 0 55596 55594 2 20 0 69796 50180 select SJ 0 0:04.53 | = | | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>>>> 0 62753 55596 0 68 0 26292 23812 urdlck IJ 0 0:00.02 | = | | `-- /usr/local/bin/dot -c >>>>> 0 37763 91496 4 29 0 15760 4696 wait I 0 0:00.05 | = | |-- sh: poudriere[main-armv7-poud-default][08]: build_pkg = (netmap-0.1.3_2) (sh) >>>>> 0 51054 37763 6 36 0 6636 3684 wait IJ 0 0:00.03 | = | | `-- /usr/bin/make -C /usr/ports/net/netmap run-depends >>>>> 0 51107 51054 3 68 0 5568 2988 wait IJ 0 0:00.01 | = | | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh >>>>> 0 51576 51107 3 68 0 11168 8860 wait IJ 0 0:00.00 | = | | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>>>> 0 51580 51576 3 20 0 68220 49432 select SJ 0 0:04.27 | = | | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>>>> 0 59063 51580 6 68 0 26292 23808 urdlck IJ 0 0:00.02 | = | | `-- /usr/local/bin/dot -c >>>>> 0 53709 91496 1 68 0 15760 4700 wait I 0 0:00.04 | = | |-- sh: poudriere[main-armv7-poud-default][06]: build_pkg = (ganv-1.8.2_1) (sh) >>>>> 0 63371 53709 6 68 0 6636 3668 wait IJ 0 0:00.03 | = | | `-- /usr/bin/make -C /usr/ports/audio/ganv lib-depends >>>>> 0 63377 63371 1 68 0 5580 2996 wait IJ 0 0:00.01 | = | | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh >>>>> 0 63413 63377 4 68 0 11180 8864 wait IJ 0 0:00.00 | = | | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>>>> 0 63414 63413 3 20 0 56212 35980 select SJ 0 0:02.21 | = | | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>>>> 0 65577 63414 6 68 0 26292 23812 urdlck IJ 0 0:00.01 | = | | `-- /usr/local/bin/dot -c >>>>> 0 63365 91496 3 68 0 15760 4696 wait I 0 0:00.03 | = | |-- sh: poudriere[main-armv7-poud-default][02]: build_pkg = (p5-GraphViz-2.25) (sh) >>>>> 0 63807 63365 7 68 0 6696 3672 wait IJ 0 0:00.02 | = | | `-- /usr/bin/make -C /usr/ports/graphics/p5-GraphViz = build-depends >>>>> 0 63808 63807 2 68 0 5568 2988 wait IJ 0 0:00.01 | = | | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh >>>>> 0 63833 63808 4 68 0 11188 8864 wait IJ 0 0:00.00 | = | | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>>>> 0 63834 63833 7 20 0 67400 48532 select SJ 0 0:03.52 | = | | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>>>> 0 66041 63834 2 68 0 26292 23812 urdlck IJ 0 0:00.02 | = | | `-- /usr/local/bin/dot -c >>>>> 0 69974 91496 1 68 0 15760 4700 wait I 0 0:00.04 | = | |-- sh: poudriere[main-armv7-poud-default][05]: build_pkg = (py311-graphviz-0.10.1) (sh) >>>>> 0 73474 69974 5 68 0 6684 3740 wait IJ 0 0:00.02 | = | | `-- /usr/bin/make -C /usr/ports/graphics/py-graphviz = FLAVOR=3Dpy311 run-depends >>>>> 0 73496 73474 6 68 0 5584 2992 wait IJ 0 0:00.01 | = | | `-- /bin/sh /usr/ports/Mk/Scripts/do-depends.sh >>>>> 0 73521 73496 7 68 0 11144 8864 wait IJ 0 0:00.00 | = | | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>>>> 0 73522 73521 0 20 0 52432 32664 select SJ 0 0:02.70 | = | | `-- /usr/local/sbin/pkg-static add -A = /packages/All/graphviz-9.0.0_4.pkg >>>>> 0 76540 73522 3 68 0 26292 23812 urdlck IJ 0 0:00.02 | = | | `-- /usr/local/bin/dot -c >>>>> 0 91907 91496 5 68 0 15760 4492 nanslp S 0 1:05.17 | = | |-- sh: poudriere[main-armv7-poud-default]: html_json_main (sh) >>>>> 0 99134 91496 1 40 0 15760 4740 piperd I 0 0:03.22 | = | `-- sh: poudriere[main-armv7-poud-default]: pkg_cacher_main (sh) >>>>> 0 23 >>>=20 >>>=20 >>> A little bit more context for /usr/local/bin/dot : >>>=20 >>> 0x20631520 in _umtx_op () from /lib/libsys.so.7 >>> (gdb) bt >>> #0 0x20631520 in _umtx_op () at /lib/libsys.so.7 >>> #1 0x2063245c in _umtx_op_err () at /lib/libsys.so.7 >>> #2 0x203a2da8 in ??? () at /lib/libthr.so.3 >>> #3 0x2039bbf4 in ??? () at /lib/libthr.so.3 >>> #4 0x20061788 in ??? () at /libexec/ld-elf.so.1 >>>=20 >>> And the associated instance of /usr/local/sbin/pkg-static : >>>=20 >>> (gdb) bt >>> #0 _poll () at _poll.S:4 >>> #1 0x007669e0 in __thr_poll (fds=3D0xd1, nfds=3D1, timeout=3D1000) = at /home/pkgbuild/worktrees/main/lib/libthr/thread/thr_syscalls.c:320 >>> #2 0x003602e8 in pkg_script_run_child (pid=3D64019, = pstat=3Dpstat@entry=3D0xffffc49c, inputfd=3D9, script_name=3D0x5d5c9 = "POST-INSTALL") at scripts.c:303 >>> #3 0x0035fc34 in pkg_script_run (pkg=3D0x20972e00, type=3D, upgrade=3D) at scripts.c:227 >>> #4 0x00371250 in pkg_add_common (db=3D, = path=3D, path@entry=3D0xffffda7f = "/packages/All/graphviz-9.0.0_4.pkg", flags=3D, = reloc=3D, remote=3D0x0, local=3D0x0, t=3D0x0) at = pkg_add.c:1386 >>> #5 0x003707e4 in pkg_add (db=3D0x4, path=3D0x1 , path@entry=3D0xffffda7f = "/packages/All/graphviz-9.0.0_4.pkg", flags=3D1000, = location=3D0x766990 <__thr_poll> "\360H-\351\020\260\215\342") at = pkg_add.c:1460 >>> #6 0x00194544 in exec_add (argc=3D, argv=3D) at add.c:178 >>> #7 0x0019f840 in main (argc=3D2, argv=3D0xffffd87c) at main.c:872 >>>=20 >>>=20 >>=20 >> Continued experiments point in a different direction >> via an simpler test of just use of "dot -c": >>=20 >> # /usr/local/bin/dot -c >> Error: /usr/local/lib/graphviz/config6 is zero sized. >>=20 >> And after the Error: line dot is hung up like it is >> when used via pkg-static. >>=20 >> It seems that whatever leads to the "Error:" line >> output and that conidition's handling in dot is the >> source of the hangup. >>=20 >> "dot -c" generates configuration file content for >> plugins and, appearently, should not gnerate an >> empty config6 file. >=20 >=20 > Well, I get to: >=20 > (gdb) bt > #0 0x2005acc0 in dlopen () from /libexec/ld-elf.so.1 > #1 0x201b87fc in vm_open (loader_data=3D, = filename=3Dfilename@entry=3D0x20662540 = "/usr/local/lib/graphviz/libgvplugin_gd.so.6", advise=3D, = advise@entry=3D0x0) > at loaders/dlopen.c:211 > #2 0x201b6f24 in tryall_dlopen (phandle=3D, = phandle@entry=3D0xffffd978, filename=3D0x20662540 = "/usr/local/lib/graphviz/libgvplugin_gd.so.6", advise=3D0x0, vtable=3D0x0)= at ltdl.c:444 > #3 0x201b52d0 in try_dlopen (phandle=3Dphandle@entry=3D0xffffd9b0, = filename=3D, filename@entry=3D0x20665040 = "/usr/local/lib/graphviz/libgvplugin_gd.so.6", ext=3D0x20662599 ".6", = advise=3D) at ltdl.c:1481 > #4 0x201b4d34 in lt_dlopenadvise (filename=3D0x20665040 = "/usr/local/lib/graphviz/libgvplugin_gd.so.6", advise=3D0x0) at = ltdl.c:1671 > #5 lt_dlopen (filename=3D0x1 ) at ltdl.c:1626 > #6 0x200e255c in ?? () >=20 > But the dlopen does not return. One possible point of interest > is that /usr/local/lib/graphviz/libgvplugin_gd.so.6 leads to > loading a bunch of libraries, including the first/only load of > /lib/libc++.so.1 and libcxxrt.so.1 ( via > /usr/local/lib/libLerc.so.4 ). Two more basic tests and relted information from an example failure: ) I replicated the problem on a RPi4B, so before any modern armv8.* . ) I mounted a stable/14 and chrooted to it but based on the main kernel I've been using. stable/14 did not repeat the problem. That last likely means that main's kernel is not the problem. It suggests code specific to main that is not in stable/14 is at issue. An example could be libsys and changes made to it during the interval between the last known working and the first known failure. A simple program source to reproduce the problem in a main armv7 chroot on a main aarch64 is: # more dlopen_test.c=20 // FAILS: // cc -g -std=3Dc11 -pedantic -Wall -pthread dlopen_test.c ; ./a.out // Works: // cc -g -std=3Dc11 -pedantic -Wall dlopen_test.c ; ./a.out #include int main(void) { // ANY OF THE FOLLOWING FAIL with -pthread specified: // = dlopen("/usr/local/lib/graphviz/libgvplugin_gd.so.6.0.0",RTLD_LAZY); // dlopen("/usr/local/lib/libpangocairo-1.0.so.0",RTLD_LAZY); dlopen("/usr/local/lib/libcairo.so.2",RTLD_LAZY); } so -pthread seems essential. # truss -fae ./a.out 13114: mmap(0x0,135168,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON,-1,0x0) = =3D 537444352 (0x2008c000) 13114: mprotect(0x2007a000,4096,PROT_READ) =3D 0 (0x0) . . . open("/lib/libthr.so.3",O_RDONLY|O_CLOEXEC|O_VERIFY,04002220025) =3D 3 = (0x3) 13114: fstat(3,{ mode=3D-r--r--r-- = ,inode=3D91507361,size=3D122044,blksize=3D32768 }) =3D 0 (0x0) 13114: mmap(0x0,4096,PROT_READ,MAP_PRIVATE|MAP_PREFAULT_READ,3,0x0) =3D = 537153536 (0x20045000) 13114: mmap(0x0,356352,PROT_NONE,MAP_GUARD,-1,0x0) =3D 537829376 = (0x200ea000) 13114: = mmap(0x200ea000,32768,PROT_READ,MAP_PRIVATE|MAP_FIXED|MAP_NOCORE|MAP_PREFA= ULT_READ,3,0x0) =3D 537829376 (0x200ea000) 13114: = mmap(0x20101000,90112,PROT_READ|PROT_EXEC,MAP_PRIVATE|MAP_FIXED|MAP_NOCORE= |MAP_PREFAULT_READ,3,0x7000) =3D 537923584 (0x20101000) 13114: = mmap(0x20126000,8192,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED|MAP_PREFAU= LT_READ,3,0x1c000) =3D 538075136 (0x20126000) 13114: = mmap(0x20137000,4096,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED|MAP_PREFAU= LT_READ,3,0x1d000) =3D 538144768 (0x20137000) 13114: = mmap(0x20138000,36864,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED|MAP_ANON,= -1,0x0) =3D 538148864 (0x20138000) 13114: munmap(0x20045000,4096) =3D 0 (0x0) 13114: close(3) =3D 0 (0x0) . . . 13114: mprotect(0x20ff0000,4096,PROT_READ) =3D 0 (0x0) 13114: mprotect(0x21025000,4096,PROT_READ) =3D 0 (0x0) 13114: mprotect(0x210a4000,4096,PROT_READ) =3D 0 (0x0) 13114: mprotect(0x210fc000,4096,PROT_READ) =3D 0 (0x0) load: 0.53 cmd: a.out 13114 [urdlck] 38.23r 0.00u 0.00s 0% 10132k #0 0xffff0000004be9b8 at mi_switch+0x184 #1 0xffff000000513880 at sleepq_switch+0xf0 #2 0xffff000000513ca8 at sleepq_catch_signals+0x2bc #3 0xffff0000005139bc at sleepq_wait_sig+0xc #4 0xffff0000004bdd54 at _sleep+0x278 #5 0xffff0000004d3088 at umtxq_sleep+0x2b0 #6 0xffff0000004daadc at do_rw_rdlock+0x36c #7 0xffff0000004d4ed8 at freebsd32__umtx_op+0x5c #8 0xffff00000086cee4 at do_el0_sync+0x5dc #9 0xffff00000084493c at handle_el0_sync+0x4c ^C13114: _umtx_op(0x20137c40,UMTX_OP_RW_RDLOCK,0x0,0x0,0x0) ERR#4 = 'Interrupted system call' 13114: SIGNAL 2 (SIGINT) code=3DSI_KERNEL 13114: process killed, signal =3D 2 Note 0x20137c40 is associated with open("/lib/libthr.so.3", . . .) activity: 13114: = mmap(0x20137000,4096,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED|MAP_PREFAU= LT_READ,3,0x1d000) =3D 538144768 (0x20137000) also, in gdb: 0x201375c0 - 0x2014092c is .bss in /lib/libthr.so.3 (gdb) bt #0 0x201aeec0 in __pthread_map_stacks_exec () from /lib/libc.so.7 #1 0x2005d1e4 in ?? () from /libexec/ld-elf.so.1 Backtrace stopped: previous frame identical to this frame (corrupt = stack?) (gdb) disass Dump of assembler code for function __pthread_map_stacks_exec: =3D> 0x201aeec0 <+0>: ldr r0, [pc, #8] @ 0x201aeed0 = <__pthread_map_stacks_exec+16> 0x201aeec4 <+4>: add r0, pc, r0 0x201aeec8 <+8>: ldr r0, [r0, #156] @ 0x9c 0x201aeecc <+12>: bx r0 0x201aeed0 <+16>: andseq r6, r7, r4, lsr #12 End of assembler dump. FYI: (gdb) run Starting program: /root/a.out =20 Catchpoint 1 Inferior loaded /lib/libgcc_s.so.1 /lib/libthr.so.3 /lib/libc.so.7 /lib/libsys.so.7 0x20058998 in r_debug_state () from /libexec/ld-elf.so.1 (gdb) bt #0 0x20058998 in r_debug_state () from /libexec/ld-elf.so.1 #1 0x2005cca4 in ?? () from /libexec/ld-elf.so.1 Backtrace stopped: previous frame identical to this frame (corrupt = stack?) (gdb) c Continuing. Catchpoint 1 Inferior loaded /usr/local/lib/libcairo.so.2 /usr/local/lib/libpixman-1.so.0 /usr/local/lib/libfontconfig.so.1 /usr/local/lib/libfreetype.so.6 /usr/local/lib/libEGL.so.1 /usr/lib/libdl.so.1 /usr/local/lib/libpng16.so.16 /usr/local/lib/libxcb-shm.so.0 /usr/local/lib/libxcb.so.1 /usr/local/lib/libxcb-render.so.0 /usr/local/lib/libXrender.so.1 /usr/local/lib/libX11.so.6 /usr/local/lib/libXext.so.6 /lib/libz.so.6 /usr/local/lib/libGL.so.1 /lib/libm.so.5 /usr/local/lib/libexpat.so.1 /usr/lib/libbz2.so.4 /usr/local/lib/libbrotlidec.so.1 /usr/local/lib/libGLdispatch.so.0 /usr/local/lib/libXau.so.6 /usr/local/lib/libXdmcp.so.6 /usr/local/lib/libGLX.so.0 /usr/local/lib/libbrotlicommon.so.1 0x20058998 in r_debug_state () from /libexec/ld-elf.so.1 (gdb) bt #0 0x20058998 in r_debug_state () from /libexec/ld-elf.so.1 #1 0x2005d184 in ?? () from /libexec/ld-elf.so.1 Backtrace stopped: previous frame identical to this frame (corrupt = stack?) (gdb) s Single stepping until exit from function r_debug_state, which has no line number information. 0x201aeec0 in __pthread_map_stacks_exec () from /lib/libc.so.7 (gdb) bt #0 0x201aeec0 in __pthread_map_stacks_exec () from /lib/libc.so.7 #1 0x2005d1e4 in ?? () from /libexec/ld-elf.so.1 Backtrace stopped: previous frame identical to this frame (corrupt = stack?) (gdb) disass Dump of assembler code for function __pthread_map_stacks_exec: =3D> 0x201aeec0 <+0>: ldr r0, [pc, #8] @ 0x201aeed0 = <__pthread_map_stacks_exec+16> 0x201aeec4 <+4>: add r0, pc, r0 0x201aeec8 <+8>: ldr r0, [r0, #156] @ 0x9c 0x201aeecc <+12>: bx r0 0x201aeed0 <+16>: andseq r6, r7, r4, lsr #12 End of assembler dump. (gdb) si 0x201aeec4 in __pthread_map_stacks_exec () from /lib/libc.so.7 (gdb) si 0x201aeec8 in __pthread_map_stacks_exec () from /lib/libc.so.7 (gdb) si 0x201aeecc in __pthread_map_stacks_exec () from /lib/libc.so.7 (gdb) si 0x20112d98 in ?? () from /lib/libthr.so.3 (gdb) bt #0 0x20112d98 in ?? () from /lib/libthr.so.3 #1 0x20059e4c in ?? () from /libexec/ld-elf.so.1 Backtrace stopped: previous frame identical to this frame (corrupt = stack?) =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sat Jul 20 08:57:26 2024 X-Original-To: arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WR0mb2D3Xz5RHFd; Sat, 20 Jul 2024 08:57:35 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4WR0mZ3QGvz4gMv; Sat, 20 Jul 2024 08:57:34 +0000 (UTC) (envelope-from kostikbel@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: from tom.home (kib@localhost [127.0.0.1] (may be forged)) by kib.kiev.ua (8.18.1/8.18.1) with ESMTP id 46K8vQvo061574; Sat, 20 Jul 2024 11:57:29 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 46K8vQvo061574 Received: (from kostik@localhost) by tom.home (8.18.1/8.18.1/Submit) id 46K8vQKv061573; Sat, 20 Jul 2024 11:57:26 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 20 Jul 2024 11:57:26 +0300 From: Konstantin Belousov To: Mark Millard Cc: arm@freebsd.org, current@freebsd.org Subject: Re: armv7-on-aarch64 stuck at urdlck: I got a replication of the "ampere2" bulk build hangup problem on a Windows DevKit 2023 Message-ID: References: <8214703E-AB28-4FB3-A3DD-03C87363D8C6@yahoo.com> List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=4.0.1 X-Spam-Checker-Version: SpamAssassin 4.0.1 (2024-03-26) on tom.home X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US] X-Rspamd-Queue-Id: 4WR0mZ3QGvz4gMv [Everything and everybody in Cc: are stripped for good]. On Fri, Jul 19, 2024 at 10:38:36PM -0700, Mark Millard wrote: > 0x201375c0 - 0x2014092c is .bss in /lib/libthr.so.3 > > (gdb) bt > #0 0x201aeec0 in __pthread_map_stacks_exec () from /lib/libc.so.7 > #1 0x2005d1e4 in ?? () from /libexec/ld-elf.so.1 > Backtrace stopped: previous frame identical to this frame (corrupt stack?) > (gdb) disass > Dump of assembler code for function __pthread_map_stacks_exec: > => 0x201aeec0 <+0>: ldr r0, [pc, #8] @ 0x201aeed0 <__pthread_map_stacks_exec+16> > 0x201aeec4 <+4>: add r0, pc, r0 > 0x201aeec8 <+8>: ldr r0, [r0, #156] @ 0x9c > 0x201aeecc <+12>: bx r0 > 0x201aeed0 <+16>: andseq r6, r7, r4, lsr #12 > End of assembler dump. > Do the following: 1. Rebuild rtld/libc/libthr with the debugging info and no optimization, i.e. ensure that flags are "-O0 -g" or "-Og -g" and not -O2. See the first comment in libexec/rtld-elf/Makefile for the hint how to do it. 2. Reproduce the issue under gdb, and backtrace all threads from userspace. I only need userspace backtrace, not either kernel-side stacks nor the syscall history. Are you sure that the issue is specific to armv7, might be it takes more efforts to reproduce on host native? From nobody Sat Jul 20 13:12:56 2024 X-Original-To: arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WR6RX2VWxz5RdY9 for ; Sat, 20 Jul 2024 13:13:12 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic305-20.consmr.mail.gq1.yahoo.com (sonic305-20.consmr.mail.gq1.yahoo.com [98.137.64.83]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4WR6RW72y0z4D4l for ; Sat, 20 Jul 2024 13:13:11 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1721481189; bh=pGkplCU33pUyAH3YIGA7w1erKUGAuLRqlirQPSDWbWg=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=ms7UNpki/W530JbpDn7PPYPizbAF9uotl5idsGdSBk1VSCSTaoI5/80IVUHz0mE7bucQgG45rYYDdgDsaVecpz/NV3GghUyK/lVKS6p0GRfCCoLThWI+7wRSWtKB03SbrJMH2XpVFHT1PInBUVXp/YNMPgQyjY4iMZK4LcW8Wnhk5ocRFpq0bScItfIIRt1NuQjY9sBRjbyfNTEpYOwwM706uSsj8GgPD6+wZo9M8F+aq22x23DsA2CbBnoRQPlhybv8Ai7BSqp1J/rt5AVhrlK1k1HiFsjjmy0uIhK12MOelt+1H4wV3WwLrR0tqBvTP/njTjow85ANyQfgK0JOGw== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1721481189; bh=q123avnTD75EvoDmRSLVBoJxeH85kuUZ1TD/9tDZPCZ=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=nmWrhDXsL6A5uSctHL493TYYnOWiJAq5tFoK48Na5+fuKBRzpCi+0MDipndPXJSAHFvW1bkOTOBS18fln0UsblJ7kVyhotLSrDRgkH/rLr8DjNvzijDOIG8Sf7x/SekawL8WRe+1BYoH7HLAky44lfPzE8hWIpCUY/a0hAhEPBtFB3ktnfYkyF92wKlrItMfyex5fOHigpBSPTbWsYzdROfXrbtxEJ7dNti4vqcT4fkogBNW2U46BeavjGjQdIc0mv7ZhjvJl8vMFKzxFH94BngfYCg6dBh7rd9GILn6+gzScCrG7x3B94qayQ0KOD0cK4Yv/78UEaw+VW/2tnSHhw== X-YMail-OSG: bLvCbvEVM1nql9IIqcAfVO2TxldBpt22szIPq4e.EVMtQGxs07zM4wwCz7kPIU8 loDMImxalGlLCKByNZrE9pMSo_OtY39zR7pBUNH0Tz1mdo9G3V6gFJVQ8U41N_ggzIcI6R7MTQyG F9BYmcCsNpPukn8SvPB6p5PfAerss.lU7d4TtVKEVUKkxiamWbeyQNkmLvI9MGhDad.mw0nZ_e_W ppofDnBqYb.NJtujY7vJchzuqtmTX9luLGe2Eb.Qq1efIX5rm98qTwSKTyjEm8u7jOpY8EJl3ye6 KKj0eJHQ0Y3_qBAS60lovvvZOet0FNAC6s.XGVmk.ceH1d8DaF5vLRngNv6a4nJ5YU_7MxM_hYg5 4csimPUPJGxH6ltxdNbVMETjN_l.Z96RXP5v4VAF4.Z2ORW8m.yOdFC8UqLFnRTEaQVpylVjui1I kPpBtIfs6A75SFgdptEV.GyyIlVLI8TlTUDizLx8Kt.rGKT2KAlZqljcz77mbdTU.msQSyAlLLi. IrLIv3J9K2_W1XM55FHaW4YPJ0QnljZnE8qYuArEg8V6jZBpdPrTK7NlC.BqB6yEmLPcSzMWkhiq nkaxI136YU0srWLQXVhJ39_XishMR.kJjtfZwxqBYabH.VntHixeHQ4b2vtJz62985aEkU1JxeEP ggLq3qOKSzUG8kXKPkV.afSV1jmjlkyC77Bk3HyI49z4fZoZYNDjglOTcgn7_aNDlS.VztR06OFw Mw2MWzvuPmRnwO7xPblESawje44LTLt.tmpHcpCUGdnQx2z9EBA0BxI51gYrEg._KoI8uJzSvvLW sY9k8R7PhGmV2VsCvWzpCygrcU3HEBvVeDOtL4zIwyelzoH3DWrllHDK3uUrwqtS1M6ys67TPLyO UkUL5sZIy9I.QEa2Lhy3KXoYdwwOs1eLu.r63sz5iJp1GdVCOD_24RFK56FiEGgpYD6GOyvfpsP8 4w06JGl_5Cvc9fWQRuPSGp4ZMbg.j8etY5EpcTPuswIzcaIkrDWz89fwPlWluZNxDhX4HHJ7Kg4f whWXyxxweiKwAjm3BfbDeTrYadjoy9VWeuG2c0w2wHuRcu3DgKWohxW0GIphkLvh9JzPZ10Hf2d3 DrdShxclwwds2Z86e4oqAglrfhScDZvq6lgvYfGZxhT4zuFOXoSBe0LGh0F85bXqFiVPMnXyHsGz wI0.y88zWdzcQxf1ItGAe5X3hrykhNtHpnTK44JEjfSTP2x8VZNuzsOEYzSUW1NXeR4su8IESkub SNkPACeMDMmCSdd0ajjLim_kKiYS0RRs8dmYMZdHTabUwqvic3SYEN9fCPNcj0nQYAiJjR9PbRrK fnsQgzXiJ4Qj4lOhqGXBboR7SpcIAWOfZs.ELgIQ7aHrpgQk3pGYDR0206Ac1rUMUX7qk_5ywU.H kd.XPSPmb.lYpm.TkZB6Xhap4xmU7UkokIoDUnYC7V7NAPfyAeNrhf1xYgKe3R0YnTLyIZ1Z1s0i MB599anYCocd5E2w4pKaHdFOuON.DFRzZgF8.03hJqpg9cTjkD52CYF8T9Xo9n2ef7ev_b.HZqK. M7n5cK67kJTHq5LJjRLTPEEArAYFEERJYy7E_EMcqLPnyldcPIANavNQOVvbh1p4ef0WpB78myZd fpTdmuRaqdqxaVWKtjKr29PRE8tP7mlLHet3bhCxRRdLCJP1DUm6hPFuzungbx_gBBz2t79eZM4e aFN6PfO5AddZ3HCsEWU_I.7hOWp9083E.wsYtwFbKvLlC67YjkSb2OHil_fVAArYykE.wQ4hL4sn c40DrHn6OW8wv9PKkjW6BdPoKQWzl07WaEsPKGwhx8Gj.OJpgzJ6VCZr2eDmY6pCt4.c8tThQUYK zUBqAIAslIeOfmad17Cj085PSz.OMbn4yiCg6TdqilS_XPgqmnLDUel3NYHE2gVahrDQuVDYqGcn 2804ugeLLwiiHauA3M1TB0qfwHFtn_Ztp0qHHnEsfDqILmK740zUWgUKwYAK47MqUh7j21o38WU6 kE_TNutPklgMCc9obiV8HahKtdS2dLZTcKCredoZmUMhzXZ5mTGmceR58lXxzWXyW5_bR_X09wku cziIwOv4rpiqRoLt3Lv_0YGdJ7MDcOIui3QjRfmHTo5whxxOVHbjXOR7XIPgxdOUTlGRagW1BxUd 21ds5TQpDVG_7eDhFpoT8pscyNVpaLKJ.y2AMfWcMIcTcoA5n9kQ5uKi5dqIIW8izX1piFGLBJRB A6ibJM7Yi8tmwFWhFaGT.rW3SHf2QAYi45tFyaAZAKbdnHbdkNPfOEW_HWPV6Y0UOwZofeQ4mdHy j1A-- X-Sonic-MF: X-Sonic-ID: 7b695fda-8793-4b27-aba0-2b4b94b3bf26 Received: from sonic.gate.mail.ne1.yahoo.com by sonic305.consmr.mail.gq1.yahoo.com with HTTP; Sat, 20 Jul 2024 13:13:09 +0000 Received: by hermes--production-gq1-799bb7c8cf-l4fvm (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 442a9f46a44ca963e9cb91b242059121; Sat, 20 Jul 2024 13:13:07 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.600.62\)) Subject: Re: armv7-on-aarch64 stuck at urdlck: I got a replication of the "ampere2" bulk build hangup problem on a Windows DevKit 2023 From: Mark Millard In-Reply-To: Date: Sat, 20 Jul 2024 06:12:56 -0700 Cc: arm@freebsd.org, current@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <3D79DF68-CDE9-497D-937F-E929C381AB8B@yahoo.com> References: <8214703E-AB28-4FB3-A3DD-03C87363D8C6@yahoo.com> To: Konstantin Belousov X-Mailer: Apple Mail (2.3774.600.62) X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] X-Rspamd-Queue-Id: 4WR6RW72y0z4D4l On Jul 20, 2024, at 01:57, Konstantin Belousov = wrote: > [Everything and everybody in Cc: are stripped for good]. >=20 > On Fri, Jul 19, 2024 at 10:38:36PM -0700, Mark Millard wrote: >> 0x201375c0 - 0x2014092c is .bss in /lib/libthr.so.3 >>=20 >> (gdb) bt >> #0 0x201aeec0 in __pthread_map_stacks_exec () from /lib/libc.so.7 >> #1 0x2005d1e4 in ?? () from /libexec/ld-elf.so.1 >> Backtrace stopped: previous frame identical to this frame (corrupt = stack?) >> (gdb) disass >> Dump of assembler code for function __pthread_map_stacks_exec: >> =3D> 0x201aeec0 <+0>: ldr r0, [pc, #8] @ 0x201aeed0 = <__pthread_map_stacks_exec+16> >> 0x201aeec4 <+4>: add r0, pc, r0 >> 0x201aeec8 <+8>: ldr r0, [r0, #156] @ 0x9c >> 0x201aeecc <+12>: bx r0 >> 0x201aeed0 <+16>: andseq r6, r7, r4, lsr #12 >> End of assembler dump. >>=20 >=20 > Do the following: > 1. Rebuild rtld/libc/libthr with the debugging info and no = optimization, > i.e. ensure that flags are "-O0 -g" or "-Og -g" and not -O2. See > the first comment in libexec/rtld-elf/Makefile for the hint how to > do it. > 2. Reproduce the issue under gdb, and backtrace all threads from = userspace. > I only need userspace backtrace, not either kernel-side stacks nor > the syscall history. The above will not happen for a while. It will be based on my personal world/kernel build context that is not a clean context. > Are you sure that the issue is specific to armv7, might be it takes = more > efforts to reproduce on host native? I do not claim to know what to vary to make aarch64 used as aarch64 a good context for concluding failure is likely impossible. I only know for the identified failure contexts for armv7 that aarch64 used as aarch64 does not fail in any testing so far. For a native armv7 example context, using /usr/local/lib/libcairo.so.2 from after installing cairo and testing on a Orange Pi+ 2ed Corext-A7 system: cc -g -std=3Dc11 -pedantic -Wall -pthread dlopen_test.c ; ./a.out fails as well (a.out hangs in urdlck STATE). The context was: # uname -apKU FreeBSD OPiP2E-RPi2v1p1 15.0-CURRENT FreeBSD 15.0-CURRENT = main-n270963-609cdb12b962 GENERIC arm armv7 1500019 1500019 from a PkgBase based installation. Note: Of the 3 .so libraries referenced in dlopen.c the /usr/local/lib/libcairo.so.2 one indirectly loads the smallest number of other libraries. So I tend to prefer to test just it when that case fails. The original problem has never been observed on ampere2 for = main-arm64-default. (So: aarch64 as aarch64.) Nor in my testing on various aarch64 systems = used as aarch64 (Cortext-A72, Cortex-A76, Cortex-A78C and Cortex-X1C mix). Nor = has aarch64 dlopen_test.c ever failed in such testing contexts. The original problem always reproduced on ampere2 for = main-armv7-default. (So aarch64 as armv7.) True as of back in late Feb and later. It always reproduces in my chroot to armv7 testing on various aarch64 systems (Cortext-A72, Cortex-A76, Cortex-A78C and Cortex-X1C mix). That includes the armv7 dlopen.c testing with one of the 3 .so's reported in the = source code. The original problem is seen via "dot -c" during graphviz installation. dlopen_test.c gets the same type of failure so far for all armv7 = execution contexts that I've tried. For reference: # more dlopen_test.c=20 // FAILS: // cc -g -std=3Dc11 -pedantic -Wall -pthread dlopen_test.c ; ./a.out // Works: // cc -g -std=3Dc11 -pedantic -Wall dlopen_test.c ; ./a.out #include int main(void) { // ANY OF THE FOLLOWING FAIL with -pthread specified: // = dlopen("/usr/local/lib/graphviz/libgvplugin_gd.so.6.0.0",RTLD_LAZY); // dlopen("/usr/local/lib/libpangocairo-1.0.so.0",RTLD_LAZY); dlopen("/usr/local/lib/libcairo.so.2",RTLD_LAZY); } Note: Successful "dot -c" activity during graphviz install activity includes loading those 3 libraries, possibly indirectly for some of the 3. The failing armv7 examples hang during a dlopen of: /usr/local/lib/graphviz/libgvplugin_gd.so.6.0.0 =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sat Jul 20 21:03:05 2024 X-Original-To: arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WRJt32lGxz5Q8XF for ; Sat, 20 Jul 2024 21:03:23 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic306-21.consmr.mail.gq1.yahoo.com (sonic306-21.consmr.mail.gq1.yahoo.com [98.137.68.84]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4WRJt30JLpz50b9 for ; Sat, 20 Jul 2024 21:03:22 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1721509400; bh=bxv0kB5D8PDMPfQfFDVaVFVuVmg4ETHZ/pxrJTPgnos=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=oPWLPDa2whbpM7rnSJ6j/SDrL09xukTn6Fh6Ok6fFbq1rZsG4UycFWXX0ltUy0WTRXMoKxzjZnuxAuHbX8zzkTfXOEiemUzsLw97TbBYMtSkLqmvam2JTtw4UumUMSLJ9YdQmihbcHMF6tkTf4CMRCsr11Bq1HM9sel7OhuWzs1kqHkVWrJgpAirfpIazIms8I2Qd6mnX8KI3/leNaLE8VFjFva+yAMNuSo94MK1LbFTfpkIqb1E7nCCTVzy25CwzFugrWeQjLs4e+iQRNPVdxao1hAXS8caCiM1LbaPZmF/F8eX5zQzLBhGmrvz8hwB1XSGWuCsNp0MQVrhfwcynQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1721509400; bh=s+8T63ZrdqgvqbM4WNWvJpXnoaGDXb5ypuBMUl+movK=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=jSfYtaa66+TVHpwjPfKT1uNAwVzGTxjPX8uvk3aADbQbPGDZOCfCVE8ftJCgTV54n18VKzsE/PDj7LRIL9jgbWgr8RuQvzlZYqIAaVFtj51NQbbpTabhpsQmmPQ6qicbAkwjLEfqOMswSAeWk6XXQtshI5eLJbN2qw1ghVoZY58DN+bDlOKUs0lAPy1XTqW5eEtDvRU6W2mBCZkuHpYQbbkwbuir7ScME1Ovge3uokICletDsTqmmfN/dkZHQpI3ICkJdz2K5Iy8H5ZCOmy6Rr3Y4aynF9BrBECbDjx/Ju4TlBf8ZR6+u3h4+VKPbQs6VB+GnVUl8nLvO4y/ZR2jNQ== X-YMail-OSG: pmMxxRoVM1lktMjPq4P6Re1HT2_e4dc5Af_0E6S2_WInSRag69i9lDzhsZqwqhi 8OC2t9.CpZD9bVZswvIax2eAuLi9Wh3f955V9ZF2YYGeodErTVItKDgWJsFQa3jZl3icQmyAbXG5 u.6L4XIxPSvx18RYfllrNOxPdcBZlQeugIwqcIuVfFEG5uNB2HlJbLtV_uvXfxFp_mG5oKtfghy7 LgQ6KOZ96Mcfw1iRoqwLTj95gSvM5ExbIBwSz2aftxjKmvVYL_F6AZH4QPSIzsWs0Ld15K_RLC3M kVEXKdjoADhJXPuXWNSd5VEZDmhsPVgw6eafselQBP9t_bdPiC2YA6z22A7wv8ydSXAsWOKQVjQ_ lx7f12feXwcG7PsvehCwFuoAxEcO6AS6kFSOEiuakqqyW6ASZr.AEt6bzkTnpyFhl228_L7qFmPR SEhRyfpPbWeUOlc70Yqi3D.IOmpcldgK2jpkgeNgfrgn0O8cFseMd5nz3tMfIKFLjDob3byawGYE aoDcXiczQRgviU9WedaQAf_xnsvuxqYqEwrRFNMJ6QqMCzis.PEkdofa6H0pbhi.svJS9hZ6dpO_ bwDWHm3b6tq8tx8_jPNj05AhnZdssNHANls5Oi5bd0vJn1mN0aCPxrg07d2l.rfIuU8GnyfCPCKq JVek9QqOO8lmcL9YRmB_OoIEyjqcZDUaIgLiFNYJxOXvVUHbrG2tj74wgubgjkgfIvkQMY6GjllS TRbU.FUEH_xFB1XEMXC5r8iVy9jV4l7Pb8IJiJXtpen_ITuuR_dqOcb158U3FfCN.Op4Ga15wWGL XH6BRpjePQHnAc9.zYF7I4o.vpkRKkMYsph_zkMKb1bSZVzLw11r5fxxXjS_M0CAUtfiriD4tGZK zyI2xPf7JT_5wffrSx2_eboGxEl6Behz59RXEkAAm0KZh4jxdiEVjumGTgNDq9zwMxZIQguq.mDA ecyp4vpOqpmbgDFLyLUswpdqynvDXELnEzaKP7JTIgPpOGx2hq8C1AEggpUjcpbE08jcfUiBw0Ra bVlvnBHuJ6gBeAwRlZLeCyD6kcE2bOOGIBgO7GsFK8MRH_E7P3uE1Mm_H2DjnIiiVpNebvax1ptV VLMIlJo4c5dP3zOkrJ4yeoyMRzKeTfc7nk789lXybkVaje07.htE0I_4P9ZQNC4ssl40CmF1AOn. mqGThqbvUHkSVL1mFBHT3Q2ncJI1HZ_KXaBvjFVdNXG8MViQCeQHKHcgUvDXiGkWLQO0BCNGCpOB a854cPDqQ1DAHOegccKaecxR_HvKw6i5c305OuHTohauaXKnmnkPCZTUIZ3IxQ43te.D36KtpGPY VlPg3Y40pk4ZHlIA0p4J4ikv2Zd5nTwYtSphIk_xX34vJcS38Xx3FTCkXQIEYdZVPhIQm.1HCSH5 edFFJUVK0HLS8ZTrBqpM2oper0ddZFpkSHoxoBT4_xQaocu6RNRuxecRdrAgS1Wi.DcL3.BHUh0Q 1.s0DQ33Jgm3wyGr9SiKhRf9ex58pgWsslqp.Sm4jb_l2og5TEqEtY1ggrCmeTDJX2R8rFL6ac4G .XgEX5cLbdm3QKasr1JC1qYz7gpepq29sjvPRGLVB_nEETRwFN7hAcMKIjG48CBHgI4KhjnJSC5F Gd3WIQqWCOBbR5bhKRw6_wGMrBa_IE66SzdO7GqiFiY8aRqd_WZbrNJH0vRe5n5MYl6FIWxmOdNK WWppgGQk3NWk1At7uVIhVIfSogr5zMPwdO7RtTrJzupiDRhxM1jj9EPwNgqSbXL7qK_fvxBsJqvG Xw4D9J99P9ehkc6FespNKfnOY_6pstE8dJntHh.JeyGo4I2nM24HNglZnQya1GV79CQ3D6nj9OZ5 xf.yfhXadU7N0KdyomU1wQzR.h0QGbqQ2uZXlvUbqWShTi0dr8ZCKJvRemhUBWKM9Udn00KolrX6 MPcO5ekwbfZPzJAhdiKe6aWigVM4anJf3bxXzCjmoSNOjye8XSGNLrSXrvkNJ6L_a1990uhAgk09 ZBpHBwsZkcy0lNyp_UA12jHS17DDeU82EVEnc2Ry4T9gdCoRi6YHJM9plqU4X_S3l6CE9Po9Z66t TzqrOD0t2Z9gVKrv9s5Zw47gIMwNkqwqr6xtIFuIwUXyf608bkMyyERTMBHJmzDeke64AboLDyoQ E.WGJU3GaEUINyu4AO.BxYIGk1apJub5RyAZWeq2o81jmILWGJ8wJiHdHCmecXwFYQfMeiYYF9Xe bJZ480KUQRnGgOyE.3ghFQ.R0vAYQu.gqFMHOyvWUx32ipK9tJRyPIeOrGjGmOcVMBf6D8zhdvVt P7Q-- X-Sonic-MF: X-Sonic-ID: a3ca7240-5aef-4767-8419-2cf13b2ecd26 Received: from sonic.gate.mail.ne1.yahoo.com by sonic306.consmr.mail.gq1.yahoo.com with HTTP; Sat, 20 Jul 2024 21:03:20 +0000 Received: by hermes--production-gq1-799bb7c8cf-dkgg4 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID f117f084ba4005eddf0c4aa47fb05f25; Sat, 20 Jul 2024 21:03:16 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.600.62\)) Subject: Re: armv7-on-aarch64 stuck at urdlck: I got a replication of the "ampere2" bulk build hangup problem on a Windows DevKit 2023 From: Mark Millard In-Reply-To: Date: Sat, 20 Jul 2024 14:03:05 -0700 Cc: arm@freebsd.org, current@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <29AD2332-4F1D-4AFD-97F8-633CA3C17179@yahoo.com> References: <8214703E-AB28-4FB3-A3DD-03C87363D8C6@yahoo.com> To: Konstantin Belousov X-Mailer: Apple Mail (2.3774.600.62) X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] X-Rspamd-Queue-Id: 4WRJt30JLpz50b9 On Jul 20, 2024, at 01:57, Konstantin Belousov = wrote: > [Everything and everybody in Cc: are stripped for good]. >=20 > On Fri, Jul 19, 2024 at 10:38:36PM -0700, Mark Millard wrote: >> 0x201375c0 - 0x2014092c is .bss in /lib/libthr.so.3 >>=20 >> (gdb) bt >> #0 0x201aeec0 in __pthread_map_stacks_exec () from /lib/libc.so.7 >> #1 0x2005d1e4 in ?? () from /libexec/ld-elf.so.1 >> Backtrace stopped: previous frame identical to this frame (corrupt = stack?) >> (gdb) disass >> Dump of assembler code for function __pthread_map_stacks_exec: >> =3D> 0x201aeec0 <+0>: ldr r0, [pc, #8] @ 0x201aeed0 = <__pthread_map_stacks_exec+16> >> 0x201aeec4 <+4>: add r0, pc, r0 >> 0x201aeec8 <+8>: ldr r0, [r0, #156] @ 0x9c >> 0x201aeecc <+12>: bx r0 >> 0x201aeed0 <+16>: andseq r6, r7, r4, lsr #12 >> End of assembler dump. >>=20 >=20 > Do the following: > 1. Rebuild rtld/libc/libthr with the debugging info and no = optimization, > i.e. ensure that flags are "-O0 -g" or "-Og -g" and not -O2. See > the first comment in libexec/rtld-elf/Makefile for the hint how to > do it. I got things like: ld: error: = /usr/obj/BUILDs/main-CA7-dbg-clang/usr/main-src/arm.armv7/lib/libcompiler_= rt/libcompiler_rt.a(absvdi2.o) is incompatible with /usr/lib/crti.o ld: error: = /usr/obj/BUILDs/main-CA7-dbg-clang/usr/main-src/arm.armv7/lib/libcompiler_= rt/libcompiler_rt.a(absvsi2.o) is incompatible with /usr/lib/crti.o . . . and: building shared library libthr.so.3 ld: error: = /usr/obj/BUILDs/main-CA7-dbg-clang/usr/main-src/arm.armv7/lib/libsys/libsy= s.so is incompatible with /usr/lib/crti.o and: ld: error: = /usr/obj/BUILDs/main-CA7-dbg-clang/usr/main-src/arm.armv7/libexec/rtld-elf= /rtld_libc.a(sigsetjmp.nossppico) is incompatible with crtbrand.o ld: error: = /usr/obj/BUILDs/main-CA7-dbg-clang/usr/main-src/arm.armv7/libexec/rtld-elf= /rtld_libc.a(lstat.nossppico) is incompatible with crtbrand.o Attempting a from-scratch buildworld with DEBUG_FLAGS=3D"-Og -g" = WITHOUT_TESTS=3Dyes on the buildworld command line ended up using -O2 instead of having -Og = -g present. Attempting with a 'env DEBUG_FLAGS=3D"-Og -g" WITHOUT_TESTS=3Dyes' = command prefix got the same. So I'm temporarily using: diff --git a/share/mk/sys.mk b/share/mk/sys.mk index 44db9266784f..9c6c7ce575a4 100644 --- a/share/mk/sys.mk +++ b/share/mk/sys.mk @@ -145,7 +145,8 @@ CC ?=3D c89 CFLAGS ?=3D -O .else CC ?=3D cc -CFLAGS ?=3D -O2 -pipe +#CFLAGS ?=3D -O2 -pipe +CFLAGS ?=3D -Og -g -pipe .if defined(NO_STRICT_ALIASING) CFLAGS +=3D -fno-strict-aliasing .endif for another from-scratch buildworld attempt. lib/libc/ and lib/libthr/ built without reporting errors and have -Og -g in the *.meta files that I looked at. But we will have to see if the rest of the buildworld makes it through without errors. > 2. Reproduce the issue under gdb, and backtrace all threads from = userspace. > I only need userspace backtrace, not either kernel-side stacks nor > the syscall history. If the buildworld completes well, it will be a while before that happens. The installation and testing would then be later. So it will be a while. > Are you sure that the issue is specific to armv7, might be it takes = more > efforts to reproduce on host native? >=20 Answered with details earlier. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sat Jul 20 23:42:23 2024 X-Original-To: arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WRNPr0pNYz5QQDX for ; Sat, 20 Jul 2024 23:42:40 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic309-21.consmr.mail.gq1.yahoo.com (sonic309-21.consmr.mail.gq1.yahoo.com [98.137.65.147]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4WRNPq5VqNz40bY for ; Sat, 20 Jul 2024 23:42:39 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1721518957; bh=BCxBVn4jZM7BPnR+68Atrn2jy1cHR5uRG5QKXgOMqBY=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=PGSudCDuG6gynSvnCi8384IDhYC0XXsLYQodWLJFDkk/CEaFejBWM9bUYYnVKOTXdwgDZ4OisZgzDWCR/tk+guga03vL4Mzxy0Jug5FhAf+OLOyIV3OaDB8uBiiRdGuKp0F45PEknz5KunBE/dJpMUyUbd8FHOTCXorxQWVZwcqA3bHioG8MyBQC4kO4P8YGJ3uHqzREr1Q2QunRCniVZHW8ak6GMTaTWbN0gBtx8YzNbH+j6kJk8oa5KKlYkDih/PHBzyFwq1XCKh0jPvMxBT1cuUe/+UnkfMzsan866ON84uKtzRUXvi5lsgjDgJLdLSmNEHwS04cUFoaknibLZg== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1721518957; bh=fP0LM3hXVSug1P+RRpHgrnFIruqXQiM1BlXoqYhUbgt=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=ZgC0o9FbAmVHvVEtD9HrsdyKWRphI0Z4CE5Ql/rfP81fbg3gnlJ4MHfsR2IcP+qZtqFjcvDzaey35DbkqaJZJkJxiQNP3d5daCpwS5gNyAi8vVT5o2S17waDcU7RdxZMaCqLGeOfOqlqJ2UHb0fMakK+vsPCtbO+A2pm2z37clyNtDAuUveaveTQG7OuHHNBh/GNWR61GEXiYIHLdVHXiQi4XR1X6pvk6d07O+/ZjqOyxEx0sbkB41fOE5q83UQMTYEeKzpeNc6ShHv/qqNmJNguaoDmUXguYj5OL8BDMBH7q1oPEqoUwVIlvueC4ealazvphfOwhIBfW1UTY3IeOQ== X-YMail-OSG: c9Fmq1cVM1mrjj0abhiLXMo_S2j9lT8IE3F5nbNaH605cDvg6xNyUOESp35hJ8J ZibHhyOuhcfJcVdjBgBt2ld2SeF2JLnyhpA.uPJoH2gmP_6tdenNE6i.60CMa97G4C4yh2RaL847 oflkji5k70_nq8QfNb4tXG9X0TwbmJbyJhymJBhre4BaCYGMFR47T1hjunaXQfKwQQNGKdELLyMT Sny.AXmvjvdnzZJMtD.XT_WmhjKwiSRyEQjRRRjNOJ5.AKJup196DM9B9h3V.W7H9h2XZCalzHA9 VBvkDAxuhQ3zLoZ0wewPm1vxLQjzmlo8ud6zZTapACKyon7uanGLpd4tADqV381FY_3T5vZnE5l0 POHH1fsoJ8tXzOlUHvU9_3QhUJHZtdRShRJyqUINUwm5LgiZV2HNPTNLnmrWM_rczIYi1dGNZoIF W20puZiTCgTLUE3NtuIdqd275c0mH1Z6I_wrzCCEq824YhMGrYCEA7oY3ldnBJQiHU2JpKS3hcPp _v.RAdVESUBz7Z2Ipd4RV.tSRkIW8XmY5.oHkBMuDACEMlzoHJ.AppeahteRKo_OLogNHk8xttoZ LiEPX0Lt.fDe74MSOjb5eMqJCw6Gcj4_QabUXUZp32U4ZtJtc2pXv8k7d6JdWChrdwcL5eRYua.Q TD6JlRF8l2FS2brgDA076Q.siUzN2oQi0IBHbkcp12ESmSFs.tmHU.3B6kPZjiiH71iPx1G3uDq6 BGiomXLgtAJM3tLYOGtH8Q9Rf5tSTMYhpWvvF7mrb2Zq0Gi.4u2g9zmFSojkkdtnfB8pSEhBqi4a 3c3SgWAx8FfCe.aEs8I88CoA8CxG9QNL_cZmkQ2t_ZEMwjCgyVke9q2eZYnesYaL614E1B8eD.nG aYxItHkmK_8s2jbmyoku_j7_b5khnkBm9I5T7Kr1jMULYhSDibd57OwLqWAGYTer_ClOt4bk6q5I tARGgwkZFXhPW1Mc.OBDhmuBswP4ACJVZIXFv3tGzrclXN1xSLp4XrGcA81r9vk62njrwKLhqVyO cEXNOLBbg3RahgqC6UwU_J99XmQOYjcpMTUjN8swJtunAV.mEPR9SfMYC9xlexBYHhf1_bKyLsQ3 YG46KfEIWRJS9ktP858M3NaR0_QxUqcJrZiURKaks9Tx0xh8B0iH61jEU8G6_j6IKSYrdIp6pg4O Ie8zUKf4vlWJJqcXTFHTFaQN8RL1xPJIQzKV_8Yi.uu73Ox1V02iw1HXKx7FEZBIVBGsf1Z9nW22 4ooFYEPE68IYLl4mjvpJJfFkRlm5pAe2jbRftjHyZ3bPeZHdAJE0jRvzPKv9q1Xl90k7MpUYLBC0 5fi1g7aklJIQuDklVUVUDLOvPCBwQLqsTT9BqVqyVg7muAelWO.dcY15cLATXIc.yIZA71gsn6j5 FIycc3oe_WEtpak9jS6IxT2yUDxGrXgJ0iz_vzjOg.RJIYj_RlbGQTutHQNhpkerx4pd8acoofPW net11qguRPbon3mEatcmm0oASaFn9UFEwMfyCXnXlNjKbWpVdo2FSawcphZCox1gEIWV80GZ6bbX DTw1hBC2.1OnEhCXzhAlSs68WazbGKIoaXqPdeXAqCTo7samTxzsptrQVPkOtIZYYigMnnCH8HRL 7K7GH8a4zdoTznMBkuUZd80jUZhINBxxPtlQxL68MG4_aBURLlaJORl2HH3mAVxPX1oMJBQQJsTf 1Rnx8jXPZQQhOuozuEeGdtnn4Gb3OV.7i1g1jQ8dzXBZzyfLN99wR.Gv2bEW2CfQZgTZBlTMU8jA QYHVpBtzOuzu4pBvYe.K1FJ1ySA9KPywmVRefjcXIR90e0.K2A9ZQCiUY3SOEFSFXjCwSCaDVDKa c4uyWiTCFxuQ_Cl_NAta5y6sxu7IalLPr89g1PDVhj9u9nkHMp86aLIpjk_HvT4Ga5oxGUpzE84k OwcKwN0AYgojAycaiFN1FFbDOngFZrxMlkf1Ws09JJaSggKAznXrA6TKgVVOfSzuI9pNKxQUJ5w1 9FAeCEtyIBlHQfx6O_732C4R1d5We3ZYWyJOdy42njb2tD7Uzy3sZoGaXiTDJ2Ed5TdwomiwM1kZ b2VVdtW6APSEFURuYw4y5ENrAYl_STMWNC6zfco7_liqhvz0R4lw0vY_F3OvfWekUGMG.XkdDOVF YmzJXyUkwr8rpyuN4z6p4e3uR0DuO7FjAdmdeXQEVYfY0mF5B61CEAW9SujG8f5FE8l1wyGLTUpp NbNC_4tmoBgZrmhYazU3Y6.lycgE4B6W9T3kgeKoqpRYpj4AdEmLCGFWj7Covs8ptasv4HYQAvcQ ivg-- X-Sonic-MF: X-Sonic-ID: 62ca2856-28ad-4821-a6b3-e31ef3e5496a Received: from sonic.gate.mail.ne1.yahoo.com by sonic309.consmr.mail.gq1.yahoo.com with HTTP; Sat, 20 Jul 2024 23:42:37 +0000 Received: by hermes--production-gq1-799bb7c8cf-l6wmw (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID b50223e1739288282ea47ac04469fa62; Sat, 20 Jul 2024 23:42:34 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.600.62\)) Subject: Re: armv7-on-aarch64 stuck at urdlck: I got a replication of the "ampere2" bulk build hangup problem on a Windows DevKit 2023 From: Mark Millard In-Reply-To: Date: Sat, 20 Jul 2024 16:42:23 -0700 Cc: arm@freebsd.org, current@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <8E9579B7-2ABF-4446-B65E-E993E7B67C5C@yahoo.com> References: <8214703E-AB28-4FB3-A3DD-03C87363D8C6@yahoo.com> To: Konstantin Belousov X-Mailer: Apple Mail (2.3774.600.62) X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] X-Rspamd-Queue-Id: 4WRNPq5VqNz40bY On Jul 20, 2024, at 01:57, Konstantin Belousov = wrote: > [Everything and everybody in Cc: are stripped for good]. >=20 > On Fri, Jul 19, 2024 at 10:38:36PM -0700, Mark Millard wrote: >> 0x201375c0 - 0x2014092c is .bss in /lib/libthr.so.3 >>=20 >> (gdb) bt >> #0 0x201aeec0 in __pthread_map_stacks_exec () from /lib/libc.so.7 >> #1 0x2005d1e4 in ?? () from /libexec/ld-elf.so.1 >> Backtrace stopped: previous frame identical to this frame (corrupt = stack?) >> (gdb) disass >> Dump of assembler code for function __pthread_map_stacks_exec: >> =3D> 0x201aeec0 <+0>: ldr r0, [pc, #8] @ 0x201aeed0 = <__pthread_map_stacks_exec+16> >> 0x201aeec4 <+4>: add r0, pc, r0 >> 0x201aeec8 <+8>: ldr r0, [r0, #156] @ 0x9c >> 0x201aeecc <+12>: bx r0 >> 0x201aeed0 <+16>: andseq r6, r7, r4, lsr #12 >> End of assembler dump. >>=20 >=20 > Do the following: > 1. Rebuild rtld/libc/libthr with the debugging info and no = optimization, > i.e. ensure that flags are "-O0 -g" or "-Og -g" and not -O2. See > the first comment in libexec/rtld-elf/Makefile for the hint how to > do it. I did a full buildworld with "-Og -g" via temporary use of: diff --git a/share/mk/sys.mk b/share/mk/sys.mk index 44db9266784f..9c6c7ce575a4 100644 --- a/share/mk/sys.mk +++ b/share/mk/sys.mk @@ -145,7 +145,8 @@ CC ?=3D c89 CFLAGS ?=3D -O .else CC ?=3D cc -CFLAGS ?=3D -O2 -pipe +#CFLAGS ?=3D -O2 -pipe +CFLAGS ?=3D -Og -g -pipe .if defined(NO_STRICT_ALIASING) CFLAGS +=3D -fno-strict-aliasing .endif I installed the result armv7 world into a directory tree and installed pkg and cairo. > 2. Reproduce the issue The dlopen_test.c based case does not fail under the world built with "-Og -g": # cc -g -std=3Dc11 -pedantic -Wall -pthread dlopen_test.c ; ./a.out #=20 > under gdb (gdb) run Starting program: /root/a.out [Inferior 1 (process 36680) exited = normally] (gdb)=20 So it does not reproduce in gdb when buildworld was based on "-Og -g". > , and backtrace all threads from userspace. > I only need userspace backtrace, not either kernel-side stacks nor > the syscall history. >=20 > Are you sure that the issue is specific to armv7, might be it takes = more > efforts to reproduce on host native? =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sun Jul 21 10:36:56 2024 X-Original-To: arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WRfx42yq1z5RFvq for ; Sun, 21 Jul 2024 10:37:12 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic301-21.consmr.mail.gq1.yahoo.com (sonic301-21.consmr.mail.gq1.yahoo.com [98.137.64.147]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4WRfx25jndz49x8 for ; Sun, 21 Jul 2024 10:37:10 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=QnTp6GTl; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.64.147 as permitted sender) smtp.mailfrom=marklmi@yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1721558228; bh=M3e/1S6z8Wc/7QF5dmL0HKBS38LJd8l7U+ufbuIrvu4=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=QnTp6GTl2jkDHMCsS88a192pYAxkzOC0L5gPv5L1MIS4IOWr/iVHCRaMSLv1V229F0RYS1QTb1HkjaAP7N/kWa6NiukJWHJ3YXLQKx5NFD4CCKxzc2JJva8yo7qwOk8AWGGLk5l7AnUNLdnVE0nJFkPl7DshQiZY/SwESoB/yhsV16aAMn2fSsw+uEAiX7/093mgTk1mS2wJ8ienSwoic4VE2aUa1JPDuzO/WNq6zOgDKEuEwTMbb2t0ARbkv8X4eIyk7+fciYLu6vZnwNpvfB1TJK98cIMTy6BOulnG5Z5LH2WzHPShqXZ4wpXpYd2LBQZAjZKuWwKuDYusnutqqw== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1721558228; bh=+wzY4/SQm4WUjbu8fQ2Pi+Bf6ip3yETzsPjYplBsBii=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=fY5oddxQtZLzQe3Tj/Pfmk3eo+tQ5VOjRKDXsCKLomaUY/Clo9rFGsMv5K0NtRvp2ZZ6x1a3xnqxqxL4VJq2tBs+sVruJrgg7jK7tPOTQqaWkG9ldc6E3bNwVbFFF0VKdNXyxZMNGvfkPxAHIbO5O7fEcBvLeGVS6h23CWra1TmFK01LnwGGhqDy7OYHRpJ+q5QlkEwUWzDwkAPpMOGTm8U85DaNe8osaFz3pKzQoe21TgKqMBJpqqJF+WLmfANRPgDK2lriIBIXsyej1m6yRdc895vKcd5GtRu121OFrTrF5KEC4CfZT85bq7x88LyRARynZX2brw/0c5aK2Joqhg== X-YMail-OSG: aaTQcYMVM1lS1OxqAr.uFTJSslnTy.jdUabfRnMKmufUB5qYEODLd9zvvNK47BX 7_6WdLkExeK6WsnoGM1L7.zDtGwluiO5RLfs4Q3GcTfonEeh5.1lJSRvDEYZNSfOVvRCpbCQGbOk 8LmqiYHOrqi4Q1QYy4A4xD1KezrzMF6W8LmV22_pZzQDZkOboK3NieHDmzTEWzRtO.mZSpT7tZ9K 7_mou23ITsgyEqEg7PMmZPF7vlDVw2HHcnhyAEw_C_BpqIcTGhUzKNbue0muxApimAgTyWjs6PM1 ApvnQqFL3de3aty.Syc_jIiwbYs88BsRb7rad6KJGRRQauNkQ5rILubSd2aszB56HDQJAIXlZrR2 KU2VuIW0SK5JUtTnfRmftL_rkXzVzuSv.zoLzQeTxm76cabBAULezQVznlmVpye1oXTeYMrPkVgP RfmQZXjO06AugiKUjZL4WODNtDEdUAfU1w.cmX.LsdjhUk19WDOHcoEgZnqoII4vt1xXA598sWcW jS31i1wP2nJQUmYqi_NkEqF9BXb7gmwDOpNNTmiukKlmdsLfPaa6wCdfCq29GUY5MFKspZhLkw9M J8HuIpY_vhZnDf1h46R2prMTE4hx5ACiPlXYQn2f1LATWEbXWqUe602o9PIhTHrzgk63.hhpwC40 uAgyDLjTyrS3.vBUKPi0OtAp3EunCgAQpQ8DM9WJDbsZrAL7TjgbNNO3lBmqXXAtSI.DSB3uXmJa im5uM.37siNKoa5Jn9qIED4ZdbOnJnmP9NfWJ677nnKbNbnHnWjNR1tI6v3hmM848dgJHBv23vfY M6bLxEEaWIejMKM5MEZWkCmxqZbc10TpAiXDUcFnOp9w3HBoV_mkpItIdsWJeZ6DQRaYfb8ILWaB VOWaaYW9bW4aZ86bTgAeMhiFb6Q1nsr7OImseVQa0zQ2NFsBk15uDZiUemdk3OV9vWBpFRP2WNPP 846AmMxC_l8x_Kk8qK0D5Xyv3DY9UVubyq1tgCtvLI_ZuOXDNLVT6.uB8PxUXRPL44wpHybhRJ6X yJp4yxo.XjpF3zY89TiUzpUF.lspKgBGoGSZBqG0bhbXDuMGyHQRTiSp3qO694FZoeb2ecIDwAMo iEhtrfqY4zjabRY4LfcDcmTb0.iBrPZT_Np_7jCO2I7Lm0aPN68LuoBMj9tZ8uC4GARb1Z5.2m92 ziRxErxRu8cdcUYUTuHvQK.w9F0Fx1gGDG5G4PWZKiqPZbrK.YKT1pUtjeu5X7C0JQkLFtDQrABp 27LY94lOR8oRMdDF4ag8EtJZEE7mU2dY.V.GQUC3shZxIxkBg9acaimUufpkHB49tYn90Xkas43j ZwcgblfUlxdE7JX7LWuX9_E54urcw3dEKvxQIZPf9rQDI1bdNcxkmwOLVuadGecfbM.m6TMBl.7x CiaGa9euEX8oJ3_6EG9toqKB6m2kL6Uf6.KZcesyZvyIuvrhO6_pdpk7fp7hu2z4QXswsysyoN8_ ivwpwPSqZGHv_wIEAH3VSBMS9krVNFWmQGZMCF9Q2m.0PNqUzgPFOi8SoJd1c0.7NKE6j3LlEfUm vzxRrnnKZ84YDzw5CvX2..Yx7oyGmKuhUROMQK6a6TOmGUCj0.Ny047dwSnz88os4JJDWGP7gvZp 9rp9qSNm3Mj2m3MpxX0ZggcZlSGs1VqBna7FjDHIH0N4Ixy2vclwhq0NIi14R7hY05enqyb_tYBG eu2tiuU98R4PFXF7UFESD8lYICaMSi51LQYPDNtnIQcMlm9UvMJ4aQ2jDu2FJcgh3xvYy9uXdicZ r0Nbkp_GWF7z_gWrpNI0jPfMlCUVoRQiUddxpZB2u264uz064UtkgxH21JChQI6NhD9.20pEVwNc fCvHKdoECGGFuuFIAEnMnRI21DP2_x3ZL_27jV7A0.2X2DRQfCHQWdKhmXJGE5Qv0ueV3HMkVSnj ICfAEb0Hdy2uQIfivSWKNcS_37WKf381ucsSc606QW_7ghYYwkHwln_jd9scZTltf0C83XBu2bCd esTfkM_9Op6NYa0iNHqccBiCbnH4lC04vPIayYejRLYsB8ljPQT6dKjATx1eRxlCBLV9iizxhAxI QDA5el32I4J4dhsA1jq5dF012j1CuUw11fE9pNFIbzKeILdtfSBgxt2gLucQSK.UdRnvY0In3xyN dONHnWkylcWFHwMska3FHwhwSTKzwbgeu7Xra3Xs0kOhpY0IzrH5qD1uP0PWyKx.cfphH0DANVPZ wgQoZAn_rfnWqHh_zBXzqSmUkkUdGZkyxkVahcgyyJmOslycyZBckcdkqs4qT.78uWGgJjvF9P.O xr.o- X-Sonic-MF: X-Sonic-ID: 2cbca218-c36d-4f45-9ef0-4e2f2dc5f3eb Received: from sonic.gate.mail.ne1.yahoo.com by sonic301.consmr.mail.gq1.yahoo.com with HTTP; Sun, 21 Jul 2024 10:37:08 +0000 Received: by hermes--production-gq1-799bb7c8cf-b6h6x (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 905ed81921bd3de58b8aa68efb769745; Sun, 21 Jul 2024 10:37:06 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.600.62\)) Subject: Re: armv7-on-aarch64 stuck at urdlck: I got a replication of the "ampere2" bulk build hangup problem on a Windows DevKit 2023 From: Mark Millard In-Reply-To: <8E9579B7-2ABF-4446-B65E-E993E7B67C5C@yahoo.com> Date: Sun, 21 Jul 2024 03:36:56 -0700 Cc: arm@freebsd.org, current@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <8214703E-AB28-4FB3-A3DD-03C87363D8C6@yahoo.com> <8E9579B7-2ABF-4446-B65E-E993E7B67C5C@yahoo.com> To: Konstantin Belousov X-Mailer: Apple Mail (2.3774.600.62) X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.72 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.72)[-0.722]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; FREEMAIL_TO(0.00)[gmail.com]; FREEMAIL_FROM(0.00)[yahoo.com]; TO_DN_SOME(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; MLMMJ_DEST(0.00)[arm@freebsd.org]; APPLE_MAILER_COMMON(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.64.147:from]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.147:from] X-Rspamd-Queue-Id: 4WRfx25jndz49x8 On Jul 20, 2024, at 16:42, Mark Millard wrote: > On Jul 20, 2024, at 01:57, Konstantin Belousov = wrote: >=20 >> [Everything and everybody in Cc: are stripped for good]. >>=20 >> On Fri, Jul 19, 2024 at 10:38:36PM -0700, Mark Millard wrote: >>> 0x201375c0 - 0x2014092c is .bss in /lib/libthr.so.3 >>>=20 >>> (gdb) bt >>> #0 0x201aeec0 in __pthread_map_stacks_exec () from /lib/libc.so.7 >>> #1 0x2005d1e4 in ?? () from /libexec/ld-elf.so.1 >>> Backtrace stopped: previous frame identical to this frame (corrupt = stack?) >>> (gdb) disass >>> Dump of assembler code for function __pthread_map_stacks_exec: >>> =3D> 0x201aeec0 <+0>: ldr r0, [pc, #8] @ 0x201aeed0 = <__pthread_map_stacks_exec+16> >>> 0x201aeec4 <+4>: add r0, pc, r0 >>> 0x201aeec8 <+8>: ldr r0, [r0, #156] @ 0x9c >>> 0x201aeecc <+12>: bx r0 >>> 0x201aeed0 <+16>: andseq r6, r7, r4, lsr #12 >>> End of assembler dump. >>>=20 >>=20 >> Do the following: >> 1. Rebuild rtld/libc/libthr with the debugging info and no = optimization, >> i.e. ensure that flags are "-O0 -g" or "-Og -g" and not -O2. See >> the first comment in libexec/rtld-elf/Makefile for the hint how to >> do it. >=20 > I did a full buildworld with "-Og -g" via temporary > use of: >=20 > diff --git a/share/mk/sys.mk b/share/mk/sys.mk > index 44db9266784f..9c6c7ce575a4 100644 > --- a/share/mk/sys.mk > +++ b/share/mk/sys.mk > @@ -145,7 +145,8 @@ CC ?=3D c89 > CFLAGS ?=3D -O > .else > CC ?=3D cc > -CFLAGS ?=3D -O2 -pipe > +#CFLAGS ?=3D -O2 -pipe > +CFLAGS ?=3D -Og -g -pipe > .if defined(NO_STRICT_ALIASING) > CFLAGS +=3D -fno-strict-aliasing > .endif >=20 > I installed the result armv7 world into a > directory tree and installed pkg and cairo. >=20 >> 2. Reproduce the issue >=20 > The dlopen_test.c based case does not fail under the world > built with "-Og -g": >=20 > # cc -g -std=3Dc11 -pedantic -Wall -pthread dlopen_test.c ; ./a.out > #=20 >=20 >> under gdb >=20 > (gdb) run > Starting program: /root/a.out [Inferior 1 (process 36680) exited = normally] > (gdb)=20 >=20 > So it does not reproduce in gdb when buildworld was based > on "-Og -g". I found another context that has useful debugger information and also fails. It avoids graphviz being involved: ) a pkgbase install that I had around (pkgbase has debug information) ) also set up /home/pkgbuild/worktrees/main/ to refer to the /usr/src/ = that pkgbase put in place ) pkg install cairo ) use of my simple dlopen program (gdb) run Starting program: /root/a.out =20 Catchpoint 7 Inferior loaded /lib/libgcc_s.so.1 /lib/libthr.so.3 /lib/libc.so.7 /lib/libsys.so.7 r_debug_state (rd=3D, m=3D) at = /home/pkgbuild/worktrees/main/libexec/rtld-elf/rtld.c:4485 4485 } (gdb) c Continuing. Breakpoint 3, get_program_var_addr (name=3D0x20042f2a "__progname", = lockstate=3D0x0) at = /home/pkgbuild/worktrees/main/libexec/rtld-elf/rtld.c:4523 4523 symlook_init(&req, name); (gdb) c Continuing. Breakpoint 3, get_program_var_addr (name=3D0x20043c97 "environ", = lockstate=3D0x0) at = /home/pkgbuild/worktrees/main/libexec/rtld-elf/rtld.c:4523 4523 symlook_init(&req, name); (gdb) c Continuing. Breakpoint 3, get_program_var_addr (name=3D0x20043c9f = "__elf_aux_vector", lockstate=3D0x0) at = /home/pkgbuild/worktrees/main/libexec/rtld-elf/rtld.c:4523 4523 symlook_init(&req, name); (gdb) c Continuing. Breakpoint 3, get_program_var_addr (name=3D0x200442e8 "__libc_atexit", = lockstate=3Dlockstate@entry=3D0xffffd668) at = /home/pkgbuild/worktrees/main/libexec/rtld-elf/rtld.c:4523 4523 symlook_init(&req, name); (gdb) c Continuing. Catchpoint 7 Inferior loaded /usr/local/lib/libcairo.so.2 /usr/local/lib/libpixman-1.so.0 /usr/local/lib/libfontconfig.so.1 /usr/local/lib/libfreetype.so.6 /usr/local/lib/libEGL.so.1 /usr/lib/libdl.so.1 /usr/local/lib/libpng16.so.16 /usr/local/lib/libxcb-shm.so.0 /usr/local/lib/libxcb.so.1 /usr/local/lib/libxcb-render.so.0 /usr/local/lib/libXrender.so.1 /usr/local/lib/libX11.so.6 /usr/local/lib/libXext.so.6 /lib/libz.so.6 /usr/local/lib/libGL.so.1 /lib/libm.so.5 /usr/local/lib/libexpat.so.1 /usr/lib/libbz2.so.4 /usr/local/lib/libbrotlidec.so.1 /usr/local/lib/libGLdispatch.so.0 /usr/local/lib/libXau.so.6 /usr/local/lib/libXdmcp.so.6 /usr/local/lib/libGLX.so.0 /usr/local/lib/libbrotlicommon.so.1 r_debug_state (rd=3D, m=3D) at = /home/pkgbuild/worktrees/main/libexec/rtld-elf/rtld.c:4485 4485 } (gdb) c Continuing. Breakpoint 3, get_program_var_addr (name=3D0x200435bf = "__pthread_map_stacks_exec", lockstate=3D0xffffd290) at = /home/pkgbuild/worktrees/main/libexec/rtld-elf/rtld.c:4523 4523 symlook_init(&req, name); (gdb) c Continuing. Breakpoint 8.3, _thr_stack_fix_protection (thrd=3D0x20070000) at = /home/pkgbuild/worktrees/main/lib/libthr/thread/thr_stack.c:140 140 round_up(thrd->attr.guardsize_attr), (gdb) bt #0 _thr_stack_fix_protection (thrd=3D0x20070000) at = /home/pkgbuild/worktrees/main/lib/libthr/thread/thr_stack.c:140 #1 __thr_map_stacks_exec () at = /home/pkgbuild/worktrees/main/lib/libthr/thread/thr_stack.c:178 #2 0x2005d1e4 in map_stacks_exec (lockstate=3D0xffffd290) at = /home/pkgbuild/worktrees/main/libexec/rtld-elf/rtld.c:5946 #3 dlopen_object (name=3Dname@entry=3D0x1042d = "/usr/local/lib/libcairo.so.2", fd=3D, fd@entry=3D-1, = refobj=3D, lo_flags=3D, mode=3D1, = lockstate=3D0xffffd290) at /home/pkgbuild/worktrees/main/libexec/rtld-elf/rtld.c:3872 #4 0x20059e4c in rtld_dlopen (name=3D0x1042d = "/usr/local/lib/libcairo.so.2", fd=3D-1, mode=3D1) at = /home/pkgbuild/worktrees/main/libexec/rtld-elf/rtld.c:3751 #5 0x00020510 in main () at dlopen_test.c:14 (gdb) s 139 mprotect((char *)thrd->attr.stackaddr_attr + (gdb) s 141 round_up(thrd->attr.stacksize_attr), (gdb) s 140 round_up(thrd->attr.guardsize_attr), (gdb) s round_up (size=3D4096) at = /home/pkgbuild/worktrees/main/lib/libthr/thread/thr_stack.c:129 129 if (size % _thr_page_size !=3D 0) (gdb) s 130 size =3D ((size / _thr_page_size) + 1) * (gdb) bt #0 round_up (size=3D4096) at = /home/pkgbuild/worktrees/main/lib/libthr/thread/thr_stack.c:130 #1 _thr_stack_fix_protection (thrd=3D0x20070000) at = /home/pkgbuild/worktrees/main/lib/libthr/thread/thr_stack.c:140 #2 __thr_map_stacks_exec () at = /home/pkgbuild/worktrees/main/lib/libthr/thread/thr_stack.c:178 #3 0x2005d1e4 in map_stacks_exec (lockstate=3D0xffffd290) at = /home/pkgbuild/worktrees/main/libexec/rtld-elf/rtld.c:5946 #4 dlopen_object (name=3Dname@entry=3D0x1042d = "/usr/local/lib/libcairo.so.2", fd=3D, fd@entry=3D-1, = refobj=3D, lo_flags=3D, mode=3D1, = lockstate=3D0xffffd290) at /home/pkgbuild/worktrees/main/libexec/rtld-elf/rtld.c:3872 #5 0x20059e4c in rtld_dlopen (name=3D0x1042d = "/usr/local/lib/libcairo.so.2", fd=3D-1, mode=3D1) at = /home/pkgbuild/worktrees/main/libexec/rtld-elf/rtld.c:3751 #6 0x00020510 in main () at dlopen_test.c:14 (gdb) si 129 if (size % _thr_page_size !=3D 0) (gdb) 130 size =3D ((size / _thr_page_size) + 1) * (gdb) bt #0 round_up (size=3D4096) at = /home/pkgbuild/worktrees/main/lib/libthr/thread/thr_stack.c:130 #1 _thr_stack_fix_protection (thrd=3D0x20070000) at = /home/pkgbuild/worktrees/main/lib/libthr/thread/thr_stack.c:140 #2 __thr_map_stacks_exec () at = /home/pkgbuild/worktrees/main/lib/libthr/thread/thr_stack.c:178 #3 0x2005d1e4 in map_stacks_exec (lockstate=3D0xffffd290) at = /home/pkgbuild/worktrees/main/libexec/rtld-elf/rtld.c:5946 #4 dlopen_object (name=3Dname@entry=3D0x1042d = "/usr/local/lib/libcairo.so.2", fd=3D, fd@entry=3D-1, = refobj=3D, lo_flags=3D, mode=3D1, = lockstate=3D0xffffd290) at /home/pkgbuild/worktrees/main/libexec/rtld-elf/rtld.c:3872 #5 0x20059e4c in rtld_dlopen (name=3D0x1042d = "/usr/local/lib/libcairo.so.2", fd=3D-1, mode=3D1) at = /home/pkgbuild/worktrees/main/libexec/rtld-elf/rtld.c:3751 #6 0x00020510 in main () at dlopen_test.c:14 (gdb) disass /s Dump of assembler code for function __thr_map_stacks_exec: . . . 130 size =3D ((size / _thr_page_size) + 1) * 0x20112eec <+340>: mov r0, r6 129 if (size % _thr_page_size !=3D 0) 0x20112ef0 <+344>: ldr r4, [pc, r4] 130 size =3D ((size / _thr_page_size) + 1) * =3D> 0x20112ef4 <+348>: mov r1, r4 0x20112ef8 <+352>: bl 0x20116b60 NOTE: 0x20116760 - 0x20116f30 is .plt in /lib/libthr.so.3 --Type for more, q to quit, c to continue without paging-- 0x20112efc <+356>: mov r9, r0 0x20112f00 <+360>: mov r0, r5 0x20112f04 <+364>: mov r1, r4 0x20112f08 <+368>: bl 0x20116b60 NOTE: 0x20116760 - 0x20116f30 is .plt in /lib/libthr.so.3 0x20112f0c <+372>: mls r1, r0, r4, r5 . . . (gdb) si 0x20112ef8 130 size =3D ((size / _thr_page_size) + 1) * (gdb) 0x20116b60 in ?? () from /lib/libthr.so.3 (gdb) bt #0 0x20116b60 in ?? () from /lib/libthr.so.3 #1 0x20112efc in round_up (size=3D4096) at = /home/pkgbuild/worktrees/main/lib/libthr/thread/thr_stack.c:130 #2 _thr_stack_fix_protection (thrd=3D0x20070000) at = /home/pkgbuild/worktrees/main/lib/libthr/thread/thr_stack.c:140 #3 __thr_map_stacks_exec () at = /home/pkgbuild/worktrees/main/lib/libthr/thread/thr_stack.c:178 #4 0x2005d1e4 in map_stacks_exec (lockstate=3D0xffffd290) at = /home/pkgbuild/worktrees/main/libexec/rtld-elf/rtld.c:5946 #5 dlopen_object (name=3Dname@entry=3D0x1042d = "/usr/local/lib/libcairo.so.2", fd=3D, fd@entry=3D-1, = refobj=3D, lo_flags=3D, mode=3D1, = lockstate=3D0xffffd290) at /home/pkgbuild/worktrees/main/libexec/rtld-elf/rtld.c:3872 #6 0x20059e4c in rtld_dlopen (name=3D0x1042d = "/usr/local/lib/libcairo.so.2", fd=3D-1, mode=3D1) at = /home/pkgbuild/worktrees/main/libexec/rtld-elf/rtld.c:3751 #7 0x00020510 in main () at dlopen_test.c:14 (gdb) si 0x20116b64 in ?? () from /lib/libthr.so.3 (gdb) si 0x20116b68 in ?? () from /lib/libthr.so.3 (gdb) si 0x20116760 in ?? () from /lib/libthr.so.3 (gdb) si 0x20116764 in ?? () from /lib/libthr.so.3 (gdb) si 0x20116768 in ?? () from /lib/libthr.so.3 (gdb) si 0x2011676c in ?? () from /lib/libthr.so.3 (gdb) si _rtld_bind_start () at = /home/pkgbuild/worktrees/main/libexec/rtld-elf/arm/rtld_start.S:78 78 stmdb sp!,{r0-r5,sl,fp} (gdb) bt #0 _rtld_bind_start () at = /home/pkgbuild/worktrees/main/libexec/rtld-elf/arm/rtld_start.S:78 #1 0x201373b0 in ?? () from /lib/libthr.so.3 NOTE: 0x201373a8 - 0x201375a0 is .got.plt in /lib/libthr.so.3 Backtrace stopped: previous frame identical to this frame (corrupt = stack?) Turns out that _thr_rtld_rlock_acquire is looping when the process is stuck: . . . (gdb) bt #0 _thr_rtld_rlock_acquire (lock=3D0x20137c40) at = /home/pkgbuild/worktrees/main/lib/libthr/thread/thr_rtld.c:121 #1 0x20060788 in rlock_acquire (lock=3D0x2008af10 , = lockstate=3Dlockstate@entry=3D0xffffd0ec) at = /home/pkgbuild/worktrees/main/libexec/rtld-elf/rtld_lock.c:259 #2 0x20059098 in _rtld_bind (obj=3D0x2008f404, reloff=3D496) at = /home/pkgbuild/worktrees/main/libexec/rtld-elf/rtld.c:1035 #3 0x2005483c in _rtld_bind_start () at = /home/pkgbuild/worktrees/main/libexec/rtld-elf/arm/rtld_start.S:89 #4 0x2005483c in _rtld_bind_start () at = /home/pkgbuild/worktrees/main/libexec/rtld-elf/arm/rtld_start.S:89 #5 0x2005483c in _rtld_bind_start () at = /home/pkgbuild/worktrees/main/libexec/rtld-elf/arm/rtld_start.S:89 . . . (gdb) info threads Id Target Id Frame * 1 LWP 100174 of process = 97711 _thr_rtld_rlock_acquire (lock=3D0x20137c40) at = /home/pkgbuild/worktrees/main/lib/libthr/thread/thr_rtld.c:121 So: Only the one main thread. It is repeating the _thr_rwlock_rdlock loop (lines 121/122): (gdb) list 115 110 _thr_rtld_rlock_acquire(void *lock) 111 { 112 struct pthread *curthread; 113 struct rtld_lock *l; 114 int errsave; 115=09 116 curthread =3D _get_curthread(); 117 SAVE_ERRNO(); 118 l =3D (struct rtld_lock *)lock; 119=09 (gdb)=20 120 THR_CRITICAL_ENTER(curthread); 121 while (_thr_rwlock_rdlock(&l->lock, 0, NULL) !=3D 0) 122 ; 123 curthread->rdlock_count++; 124 RESTORE_ERRNO(); 125 } >> , and backtrace all threads from userspace. >> I only need userspace backtrace, not either kernel-side stacks nor >> the syscall history. >>=20 >> Are you sure that the issue is specific to armv7, might be it takes = more >> efforts to reproduce on host native? =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sun Jul 21 21:00:41 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WRwmV1QP0z5QV98 for ; Sun, 21 Jul 2024 21:00:42 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WRwmT1ylGz4K1n for ; Sun, 21 Jul 2024 21:00:41 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1721595641; a=rsa-sha256; cv=none; b=mgPKU1EYJ1qsQfy+ZgOQsMHxp5HdmpclZ2I6LPbzYspyRdBUmP2guYPYwuXAGNZBRdaIYl YJh8NNh68HWNk9eaYVYIwjA/M+jIw+XvQqmDwG/CR81gjp+L0t8ZQQT7WQwD5lXe7hECbm jrS3/ZxaL+8BxPhh5olruzX8jYjfagr3G0Zx55CG4KTebmawWVsCosSNNQW4/4TMUDd0tX KrH0Bk3lyiU8KAOKYAdgC7wNOAP9fgamm/44JX31RFVAYnW3T/V1eSCH652o6n70Ih18Ta X6dqPogX/FEpcbOMeS153Ya42sz2SOSA3KCWzSiMsv6gfeXIWawYeRFJo58bYg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1721595641; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=QMTpyT+n5o296g5r6H3yZHM4Xsu2BQnbg6JxRxwBMNs=; b=Ms1kppZVzrJJtLggVMClSQAFIAA+eg1u3lt502FfvgNN1FkOi4qFcZYCX7G2kc1rs0085A IV5SuKpRKgO8C32huOKbvbRNVaIvO5Soj1YXzZQVJqZerhiOCbtaTYqneuNKh9brdTK6cv 7MZNJ74SdEXAXpFkpJanJp38WQGn7roKklaKTrIXum1jY6BZY1gXeJLE2x3ANy9gFIczQG kJL+LbMaKS4OEe15c6tmedKDOmhV9bLRPqYQiL+cxAjAl7CBhgDSelsoGx7LBGLh0JKdP7 /b2gpXptcB8WmoE77LsoKRlYObsOtlVMqDyPBFKi6oU6rpuFN1vscdm0MU5vvQ== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4WRwmT1Zy3zGQD for ; Sun, 21 Jul 2024 21:00:41 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 46LL0fV1019330 for ; Sun, 21 Jul 2024 21:00:41 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 46LL0fPG019329 for freebsd-arm@FreeBSD.org; Sun, 21 Jul 2024 21:00:41 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <202407212100.46LL0fPG019329@kenobi.freebsd.org> X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@FreeBSD.org using -f From: bugzilla-noreply@FreeBSD.org To: freebsd-arm@FreeBSD.org Subject: Problem reports for freebsd-arm@FreeBSD.org that need special attention Date: Sun, 21 Jul 2024 21:00:41 +0000 List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@FreeBSD.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="17215956410.Db79.12228" Content-Transfer-Encoding: 7bit --17215956410.Db79.12228 Date: Sun, 21 Jul 2024 21:00:41 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- Open | 264574 | sdhci(4): Support ACPI attachment in BCM2835_sdhc 1 problems total for which you should take action. --17215956410.Db79.12228 Date: Sun, 21 Jul 2024 21:00:41 +0000 MIME-Version: 1.0 Content-Type: text/html; charset="UTF-8" 
The following is a listing of current problems submitted by FreeBSD users,
which need special attention. These represent problem reports covering
all versions including experimental development code and obsolete releases.

Status      |    Bug Id | Description
------------+-----------+---------------------------------------------------
Open        |    264574 | sdhci(4): Support ACPI attachment in BCM2835_sdhc

1 problems total for which you should take action.
--17215956410.Db79.12228--