From nobody Wed Nov 8 19:54:33 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SQbS5445Zz4ywB3 for ; Wed, 8 Nov 2023 19:56:05 +0000 (UTC) (envelope-from csgordon@fastmail.com) Received: from wout1-smtp.messagingengine.com (wout1-smtp.messagingengine.com [64.147.123.24]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SQbS45wxzz4Fqv for ; Wed, 8 Nov 2023 19:56:04 +0000 (UTC) (envelope-from csgordon@fastmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=fastmail.com header.s=fm3 header.b=GgRPqXjv; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=utubM6em; spf=pass (mx1.freebsd.org: domain of csgordon@fastmail.com designates 64.147.123.24 as permitted sender) smtp.mailfrom=csgordon@fastmail.com; dmarc=pass (policy=none) header.from=fastmail.com Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.west.internal (Postfix) with ESMTP id 31ED632009EC for ; Wed, 8 Nov 2023 14:56:01 -0500 (EST) Received: from imap53 ([10.202.2.103]) by compute2.internal (MEProxy); Wed, 08 Nov 2023 14:56:01 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastmail.com; h= cc:content-type:content-type:date:date:from:from:in-reply-to :message-id:mime-version:reply-to:sender:subject:subject:to:to; s=fm3; t=1699473360; x=1699559760; bh=Kh+R3kJ+1tEH5g0f7ngxRnTuH R86vJcx4m2YC9zcn90=; b=GgRPqXjvdcFmTUqq/7Jc/aIwH48o3L3SZjyOjGo6b rSJIsExggVOrlDWyNjJ8Gq3mCLJe93uejnAWZAduqbFVCXO5DD2yAVrkRuBhV8BW QHj566VsO3reLVejq2F4uvYZaqVESxDJqxO/D3PfokY2pWgXO7+L+m74elPJCmd7 /jLdrk/Ukdr1tfMhXCZUhhaIUn7OTGD6T2uykTmYn909ItIxPm0lfcQuK0ZvxQ2i xQRy7MA9tWxHKlMCPRWi9FVS5QmFXAbuxOQZcArmJhDBn/a/d69ADCG7VkTjwtIO 6rOT/VojKutRTyoj1CAx4AcMRb20dvci1vagfrbgqV4Mg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:message-id :mime-version:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t= 1699473360; x=1699559760; bh=Kh+R3kJ+1tEH5g0f7ngxRnTuHR86vJcx4m2 YC9zcn90=; b=utubM6empgh52eb5BEsMY++YdkZJmi0Jgb+tR15OpUSYgz3W7uO K1i6crrrPecLoEDaZuibvCR4ULYmzOk/jQEUDWVme1whs0izTV85YhR5F7AeUb5x YlSBoOmTBrLBvJAgJEz6QEblaspgM4V7MKPYfjTpJ2E7Sg05R/vqhDYuAsSCgPqF pOjTwDWyeOvVtTNreOwQc3XPo3lyKTK9LvcY18WpJMycWBeUuLrTZ8uZWjhChFgl fqR7rluVXoLfDgHR1c18FEtHIktPs+3i9756/VYZVvn1oE6N0h6xEJeoFYHE7nzl e+5c80fB9yoPnasPCb6Uhxk29mfnNzgWPyw== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrudduledguddvlecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecunecujfgurhepofgfggfkfffhvffutgesrgdtre erreertdenucfhrhhomhepfdevohhlihhnucfurdcuifhorhguohhnfdcuoegtshhgohhr ughonhesfhgrshhtmhgrihhlrdgtohhmqeenucggtffrrghtthgvrhhnpeekkeejieeuie elleetjeekfeetveehudfgffetgeelffefleehhfettdefvdevgeenucffohhmrghinhep fhhrvggvsghsugdrohhrghdpghhithhhuhgsrdgtohhmnecuvehluhhsthgvrhfuihiivg eptdenucfrrghrrghmpehmrghilhhfrhhomheptghsghhorhguohhnsehfrghsthhmrghi lhdrtghomh X-ME-Proxy: Feedback-ID: i60c149f6:Fastmail Received: by mailuser.nyi.internal (Postfix, from userid 501) id 63BE03640069; Wed, 8 Nov 2023 14:56:00 -0500 (EST) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.9.0-alpha0-1108-g3a29173c6d-fm-20231031.005-g3a29173c List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Message-Id: Date: Wed, 08 Nov 2023 14:54:33 -0500 From: "Colin S. Gordon" To: freebsd-hackers@FreeBSD.org Subject: Help cross-building kernel from macos Content-Type: multipart/alternative; boundary=cc114965f19149b8a12b00c0d4ca10ba X-Spamd-Result: default: False [-4.44 / 15.00]; URI_HIDDEN_PATH(1.00)[http://usr.bin/clang/llvm-tblgen/.depend]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.95)[-0.946]; DMARC_POLICY_ALLOW(-0.50)[fastmail.com,none]; RWL_MAILSPIKE_EXCELLENT(-0.40)[64.147.123.24:from]; R_DKIM_ALLOW(-0.20)[fastmail.com:s=fm3,messagingengine.com:s=fm3]; R_SPF_ALLOW(-0.20)[+ip4:64.147.123.24]; RCVD_IN_DNSWL_LOW(-0.10)[64.147.123.24:from]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; XM_UA_NO_VERSION(0.01)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; ARC_NA(0.00)[]; ASN(0.00)[asn:29838, ipnet:64.147.123.0/24, country:US]; FREEMAIL_ENVFROM(0.00)[fastmail.com]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_NONE(0.00)[]; FREEMAIL_FROM(0.00)[fastmail.com]; RCVD_TLS_LAST(0.00)[]; MID_RHS_MATCH_FROMTLD(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[fastmail.com:+,messagingengine.com:+]; MIME_TRACE(0.00)[0:+,1:+,2:~]; MLMMJ_DEST(0.00)[freebsd-hackers@FreeBSD.org] X-Rspamd-Queue-Id: 4SQbS45wxzz4Fqv X-Spamd-Bar: ---- --cc114965f19149b8a12b00c0d4ca10ba Content-Type: text/plain This is a longshot but: I'm trying to get cross-building working from my mac so I don't have to lug 2 laptops around every day to make any progress on some driver work, but so far haven't had too much luck. This is (to my pleasant surprise!) supported (https://docs.freebsd.org/en/books/handbook/cutting-edge/#building-on-non-freebsd-hosts) and as Ed pointed out on Mastodon, even runs successfully in CI (https://github.com/freebsd/freebsd-src/blob/main/.github/workflows/cross-bootstrap-tools.yml). But so far I'm unable get this working on my own machine. I was initially trying to get this running with my slightly-stale branch where I'm doing some RISC-V stuff, but hit the same problem building a recent commit from HEAD that does cross-build from macos in CI, as evidenced here: https://github.com/freebsd/freebsd-src/actions/runs/6801518714/job/18492566834 I've installed the same dependencies (same version of LLVM) specified in the github action file, and am building with the same commands (with different local directories, of course). I appear to be one macos release ahead of the version used in CI, though it doesn't seem like that should affect the failure I'm encountering. The first command, MAKEOBJDIRPREFIX=~/freebsd-obj/ tools/build/make.py --debug --cross-bindir=/usr/local/opt/llvm@13/bin TARGET=riscv TARGET_ARCH=riscv64 -n seems to work fine. But the second, a pre-requisite to building the kernel, fails: MAKEOBJDIRPREFIX=~/freebsd-obj/ tools/build/make.py --debug --cross-bindir=/usr/local/opt/llvm@13/bin TARGET=riscv TARGET_ARCH=riscv64 kernel-toolchain -s -DWITH_DISK_IMAGE_TOOLS_BOOTSTRAP Inferred CC as /usr/bin/cc Running ['/usr/bin/cc', '--version'] Apple clang version 14.0.3 (clang-1403.0.22.14.1) Target: x86_64-apple-darwin22.6.0 Thread model: posix InstalledDir: /Library/Developer/CommandLineTools/usr/bin Inferred CXX as /usr/bin/c++ Running ['/usr/bin/c++', '--version'] Apple clang version 14.0.3 (clang-1403.0.22.14.1) Target: x86_64-apple-darwin22.6.0 Thread model: posix InstalledDir: /Library/Developer/CommandLineTools/usr/bin Inferred CPP as /usr/bin/cpp Running ['/usr/bin/cpp', '--version'] Apple clang version 14.0.3 (clang-1403.0.22.14.1) Target: x86_64-apple-darwin22.6.0 Thread model: posix InstalledDir: /Library/Developer/CommandLineTools/usr/bin Inferred XCC as /usr/local/opt/llvm@13/bin/clang Running ['/usr/local/opt/llvm@13/bin/clang', '--version'] Homebrew clang version 13.0.1 Target: x86_64-apple-darwin22.6.0 Thread model: posix InstalledDir: /usr/local/opt/llvm@13/bin Inferred XCXX as /usr/local/opt/llvm@13/bin/clang++ Running ['/usr/local/opt/llvm@13/bin/clang++', '--version'] Homebrew clang version 13.0.1 Target: x86_64-apple-darwin22.6.0 Thread model: posix InstalledDir: /usr/local/opt/llvm@13/bin Inferred XCPP as /usr/local/opt/llvm@13/bin/clang-cpp Running ['/usr/local/opt/llvm@13/bin/clang-cpp', '--version'] Homebrew clang version 13.0.1 Target: x86_64-apple-darwin22.6.0 Thread model: posix InstalledDir: /usr/local/opt/llvm@13/bin Inferred XLD as /usr/local/opt/llvm@13/bin/ld.lld Running ['/usr/local/opt/llvm@13/bin/ld.lld', '--version'] Homebrew LLD 13.0.1 (compatible with GNU linkers) Source bmake version: 20230909 Installed bmake version: 20230909 Configure args: --with-default-sys-path=.../share/mk:/Users/.../freebsd-obj/bmake-install/share/mk --with-machine=amd64 --without-filemon --prefix=/Users/.../freebsd-obj/bmake-install Last configure args: --with-default-sys-path=.../share/mk:/Users/.../freebsd-obj/bmake-install/share/mk --with-machine=amd64 --without-filemon --prefix=/Users/.../freebsd-obj/bmake-install Adding -DWITH_AUTO_OBJ Running `env CC=/usr/bin/cc CXX=/usr/bin/c++ CPP=/usr/bin/cpp XCC=/usr/local/opt/llvm@13/bin/clang XCXX=/usr/local/opt/llvm@13/bin/clang++ XCPP=/usr/local/opt/llvm@13/bin/clang-cpp XLD=/usr/local/opt/llvm@13/bin/ld.lld /Users/.../freebsd-obj/bmake-install/bin/bmake TARGET=riscv TARGET_ARCH=riscv64 kernel-toolchain -s -DWITH_DISK_IMAGE_TOOLS_BOOTSTRAP -DWITH_AUTO_OBJ -DWITHOUT_CLEAN` >>> Deleting stale files in build tree... 6.29 real 4.95 user 1.61 sys -------------------------------------------------------------- >>> Rebuilding the temporary build tree -------------------------------------------------------------- Linking host tools into /Users/.../freebsd-obj/Users/.../freebsd-src/riscv.riscv64/tmp/legacy/bin -------------------------------------------------------------- >>> stage 1.1: legacy release compatibility shims -------------------------------------------------------------- ===> tools/build (obj,includes,all,install) real 0m1.938s user 0m0.624s sys 0m0.697s -------------------------------------------------------------- >>> stage 1.2: bootstrap tools -------------------------------------------------------------- >>> Deleting stale dependencies... real 0m0.385s user 0m0.132s sys 0m0.211s ===> lib/clang/libllvmminimal (obj,all,install) ===> usr.bin/clang/llvm-tblgen (obj,all,install) bmake[3]: /Users/.../freebsd-obj/Users/.../freebsd-src/riscv.riscv64/tmp/obj-tools/usr.bin/clang/llvm-tblgen/.depend, 1: ignoring stale .depend for /Users/.../freebsd-obj//Users/.../freebsd-src/riscv.riscv64/tmp/legacy/usr/lib/libz.a bmake[3]: /Users/.../freebsd-obj/Users/.../freebsd-src/riscv.riscv64/tmp/obj-tools/usr.bin/clang/llvm-tblgen/.depend, 1: ignoring stale .depend for /usr/lib/libprivatezstd.a ld: warning: directory not found for option '-L/Users/.../freebsd-obj/Users/.../freebsd-src/riscv.riscv64/tmp/obj-tools/lib/libthr' ld: warning: directory not found for option '-L/Users/.../freebsd-obj/Users/.../freebsd-src/riscv.riscv64/tmp/obj-tools/lib/libz' ld: warning: directory not found for option '-L/Users/.../freebsd-obj/Users/.../freebsd-src/riscv.riscv64/tmp/obj-tools/lib/libthr' ld: library not found for -lprivatezstd clang: error: linker command failed with exit code 1 (use -v to see invocation) *** Error code 1 Stop. bmake[3]: stopped in /Users/.../freebsd-src/usr.bin/clang/llvm-tblgen *** Error code 1 Stop. bmake[2]: stopped in /Users/.../freebsd-src *** Error code 1 Stop. bmake[1]: stopped in /Users/.../freebsd-src *** Error code 1 Stop. bmake: stopped in /Users/.../freebsd-src I get the same failure if I just try to buildworld instead of build kernel-toolchain. Above I'm trying to cross-build riscv64, but the same things happens when I try other platforms as well (e.g., aarch64, which specifically is built in CI). I figure relatively few people have actually debugged a cross-build from macos, but has anyone seen similar build errors about privatezstd before, or other llvm-tblgen build failures? Thanks for any pointers, Colin --cc114965f19149b8a12b00c0d4ca10ba Content-Type: text/html Content-Transfer-Encoding: quoted-printable
This is a longs= hot but: I'm trying to get cross-building working from my mac so I don't= have to lug 2 laptops around every day to make any progress on some dri= ver work, but so far haven't had too much luck.  This is (to my ple= asant surprise!) supported (https://docs.free= bsd.org/en/books/handbook/cutting-edge/#building-on-non-freebsd-hosts) and as Ed pointed out on Mastodon, even runs successfully in CI (https://github.com/freebsd/freebsd-src/blob= /main/.github/workflows/cross-bootstrap-tools.yml). But so far I'm u= nable get this working on my own machine.  I was initially trying t= o get this running with my slightly-stale branch where I'm doing some RI= SC-V stuff, but hit the same problem building a recent commit from HEAD = that does cross-build from macos in CI, as evidenced here: https://github.com/freebsd/freebsd-src/actions/runs/6801518714/job= /18492566834

I've installed the same de= pendencies (same version of LLVM) specified in the github action file, a= nd am building with the same commands (with different local directories,= of course).  I appear to be one macos release ahead of the version= used in CI, though it doesn't seem like that should affect the failure = I'm encountering.  The first command,

=     MAKEOBJDIRPREFIX=3D~/freebsd-obj/ tools/build/make.py= --debug --cross-bindir=3D/usr/local/opt/llvm@13/bin TARGET=3Driscv TARG= ET_ARCH=3Driscv64 -n

seems to work fine. Bu= t the second, a pre-requisite to building the kernel, fails:

MAKEOBJDIRPREFIX=3D~/freebsd-obj/ tools/build/make.py = --debug --cross-bindir=3D/usr/local/opt/llvm@13/bin TARGET=3Driscv TARGE= T_ARCH=3Driscv64 kernel-toolchain -s -DWITH_DISK_IMAGE_TOOLS_BOOTSTRAP
Inferred CC as /usr/bin/cc
Running ['/usr/bin= /cc', '--version']
Apple clang version 14.0.3 (clang-1403.= 0.22.14.1)
Target: x86_64-apple-darwin22.6.0
Thread model: posix
InstalledDir: /Library/Developer/Comm= andLineTools/usr/bin
Inferred CXX as /usr/bin/c++
Running ['/usr/bin/c++', '--version']
Apple clang ve= rsion 14.0.3 (clang-1403.0.22.14.1)
Target: x86_64-apple-d= arwin22.6.0
Thread model: posix
InstalledDir= : /Library/Developer/CommandLineTools/usr/bin
Inferred CPP= as /usr/bin/cpp
Running ['/usr/bin/cpp', '--version']
=
Apple clang version 14.0.3 (clang-1403.0.22.14.1)
Target: x86_64-apple-darwin22.6.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin
Inferred XCC as /usr/local/opt/llvm@13/bin/clang
<= div>Running ['/usr/local/opt/llvm@13/bin/clang', '--version']
<= div>Homebrew clang version 13.0.1
Target: x86_64-apple-dar= win22.6.0
Thread model: posix
InstalledDir: = /usr/local/opt/llvm@13/bin
Inferred XCXX as /usr/local/opt= /llvm@13/bin/clang++
Running ['/usr/local/opt/llvm@13/bin/= clang++', '--version']
Homebrew clang version 13.0.1
Target: x86_64-apple-darwin22.6.0
Thread model: p= osix
InstalledDir: /usr/local/opt/llvm@13/bin
Inferred XCPP as /usr/local/opt/llvm@13/bin/clang-cpp
Ru= nning ['/usr/local/opt/llvm@13/bin/clang-cpp', '--version']
Homebrew clang version 13.0.1
Target: x86_64-apple-darwi= n22.6.0
Thread model: posix
InstalledDir: /u= sr/local/opt/llvm@13/bin
Inferred XLD as /usr/local/opt/ll= vm@13/bin/ld.lld
Running ['/usr/local/opt/llvm@13/bin/ld.l= ld', '--version']
Homebrew LLD 13.0.1 (compatible with GNU= linkers)
Source bmake version: 20230909
Ins= talled bmake version: 20230909
Configure args: --with-defa= ult-sys-path=3D.../share/mk:/Users/.../freebsd-obj/bmake-install/share/m= k --with-machine=3Damd64 --without-filemon --prefix=3D/Users/.../freebsd= -obj/bmake-install
Last configure args: --with-default-sys= -path=3D.../share/mk:/Users/.../freebsd-obj/bmake-install/share/mk --wit= h-machine=3Damd64 --without-filemon --prefix=3D/Users/.../freebsd-obj/bm= ake-install
Adding -DWITH_AUTO_OBJ
Running `= env CC=3D/usr/bin/cc CXX=3D/usr/bin/c++ CPP=3D/usr/bin/cpp XCC=3D/usr/lo= cal/opt/llvm@13/bin/clang XCXX=3D/usr/local/opt/llvm@13/bin/clang++ XCPP= =3D/usr/local/opt/llvm@13/bin/clang-cpp XLD=3D/usr/local/opt/llvm@13/bin= /ld.lld /Users/.../freebsd-obj/bmake-install/bin/bmake TARGET=3Driscv TA= RGET_ARCH=3Driscv64 kernel-toolchain -s -DWITH_DISK_IMAGE_TOOLS_BOOTSTRA= P -DWITH_AUTO_OBJ -DWITHOUT_CLEAN`
>>> Deleting s= tale files in build tree...
     =    6.29 real         4= .95 user         1.61 sys

-------------------------------------------------= -------------
>>> Rebuilding the temporary build = tree
-----------------------------------------------------= ---------
Linking host tools into /Users/.../freebsd-obj/U= sers/.../freebsd-src/riscv.riscv64/tmp/legacy/bin

--------------------------------------------------------------
>>> stage 1.1: legacy release compatibility shims
-----------------------------------------------------------= ---
=3D=3D=3D> tools/build (obj,includes,all,install)

real 0m1.938s
user 0m0.624s
sys 0m0.697s

------------------= --------------------------------------------
>>> = stage 1.2: bootstrap tools
-------------------------------= -------------------------------
>>> Deleting stal= e dependencies...

real 0m0.385s
user 0m0.132s
sys 0m0.211s
=3D=3D=3D> = lib/clang/libllvmminimal (obj,all,install)
=3D=3D=3D> <= a href=3D"http://usr.bin/clang/llvm-tblgen">usr.bin/clang/llvm-tblgen (obj,all,install)
bmake[3]: /Users/.../freebsd-obj/Users= /.../freebsd-src/riscv.riscv64/tmp/obj-tools/usr.bin/clang/llvm-tblgen/.depend, 1: igno= ring stale .depend for /Users/.../freebsd-obj//Users/.../freebsd-src/ris= cv.riscv64/tmp/legacy/usr/lib/libz.a
bmake[3]: /Users/.../= freebsd-obj/Users/.../freebsd-src/riscv.riscv64/tmp/obj-tools/usr.bin/clang/llvm-tblgen/.de= pend, 1: ignoring stale .depend for /usr/lib/libprivatezstd.a
ld: warning: directory not found for option '-L/Users/.../freebs= d-obj/Users/.../freebsd-src/riscv.riscv64/tmp/obj-tools/lib/libthr'
<= /div>
ld: warning: directory not found for option '-L/Users/.../free= bsd-obj/Users/.../freebsd-src/riscv.riscv64/tmp/obj-tools/lib/libz'
<= /div>
ld: warning: directory not found for option '-L/Users/.../free= bsd-obj/Users/.../freebsd-src/riscv.riscv64/tmp/obj-tools/lib/libthr'
ld: library not found for -lprivatezstd
clang:= error: linker command failed with exit code 1 (use -v to see invocation= )
*** Error code 1

Stop.
<= /div>
bmake[3]: stopped in /Users/.../freebsd-src/usr.bin/clang/llvm-tblgen
*= ** Error code 1

Stop.
bmake[2= ]: stopped in /Users/.../freebsd-src
*** Error code 1
<= /div>

Stop.
bmake[1]: stopped in /Users= /.../freebsd-src
*** Error code 1

=
Stop.
bmake: stopped in /Users/.../freebsd-src


I get the same failure if I just t= ry to buildworld instead of build kernel-toolchain. Above I'm trying to = cross-build riscv64, but the same things happens when I try other platfo= rms as well (e.g., aarch64, which specifically is built in CI).

I figure relatively few people have actually debugg= ed a cross-build from macos, but has anyone seen similar build errors ab= out privatezstd before, or other llvm-tblgen build failures?

Thanks for any pointers,
Colin
= --cc114965f19149b8a12b00c0d4ca10ba--