Date: Wed, 08 Nov 2023 14:54:33 -0500 From: "Colin S. Gordon" <csgordon@fastmail.com> To: freebsd-hackers@FreeBSD.org Subject: Help cross-building kernel from macos Message-ID: <f42da895-b8e3-49a8-826a-d448103b047b@app.fastmail.com>
next in thread | raw e-mail | index | archive | help
--cc114965f19149b8a12b00c0d4ca10ba Content-Type: text/plain This is a longshot but: I'm trying to get cross-building working from my mac so I don't have to lug 2 laptops around every day to make any progress on some driver work, but so far haven't had too much luck. This is (to my pleasant surprise!) supported (https://docs.freebsd.org/en/books/handbook/cutting-edge/#building-on-non-freebsd-hosts) and as Ed pointed out on Mastodon, even runs successfully in CI (https://github.com/freebsd/freebsd-src/blob/main/.github/workflows/cross-bootstrap-tools.yml). But so far I'm unable get this working on my own machine. I was initially trying to get this running with my slightly-stale branch where I'm doing some RISC-V stuff, but hit the same problem building a recent commit from HEAD that does cross-build from macos in CI, as evidenced here: https://github.com/freebsd/freebsd-src/actions/runs/6801518714/job/18492566834 I've installed the same dependencies (same version of LLVM) specified in the github action file, and am building with the same commands (with different local directories, of course). I appear to be one macos release ahead of the version used in CI, though it doesn't seem like that should affect the failure I'm encountering. The first command, MAKEOBJDIRPREFIX=~/freebsd-obj/ tools/build/make.py --debug --cross-bindir=/usr/local/opt/llvm@13/bin TARGET=riscv TARGET_ARCH=riscv64 -n seems to work fine. But the second, a pre-requisite to building the kernel, fails: MAKEOBJDIRPREFIX=~/freebsd-obj/ tools/build/make.py --debug --cross-bindir=/usr/local/opt/llvm@13/bin TARGET=riscv TARGET_ARCH=riscv64 kernel-toolchain -s -DWITH_DISK_IMAGE_TOOLS_BOOTSTRAP Inferred CC as /usr/bin/cc Running ['/usr/bin/cc', '--version'] Apple clang version 14.0.3 (clang-1403.0.22.14.1) Target: x86_64-apple-darwin22.6.0 Thread model: posix InstalledDir: /Library/Developer/CommandLineTools/usr/bin Inferred CXX as /usr/bin/c++ Running ['/usr/bin/c++', '--version'] Apple clang version 14.0.3 (clang-1403.0.22.14.1) Target: x86_64-apple-darwin22.6.0 Thread model: posix InstalledDir: /Library/Developer/CommandLineTools/usr/bin Inferred CPP as /usr/bin/cpp Running ['/usr/bin/cpp', '--version'] Apple clang version 14.0.3 (clang-1403.0.22.14.1) Target: x86_64-apple-darwin22.6.0 Thread model: posix InstalledDir: /Library/Developer/CommandLineTools/usr/bin Inferred XCC as /usr/local/opt/llvm@13/bin/clang Running ['/usr/local/opt/llvm@13/bin/clang', '--version'] Homebrew clang version 13.0.1 Target: x86_64-apple-darwin22.6.0 Thread model: posix InstalledDir: /usr/local/opt/llvm@13/bin Inferred XCXX as /usr/local/opt/llvm@13/bin/clang++ Running ['/usr/local/opt/llvm@13/bin/clang++', '--version'] Homebrew clang version 13.0.1 Target: x86_64-apple-darwin22.6.0 Thread model: posix InstalledDir: /usr/local/opt/llvm@13/bin Inferred XCPP as /usr/local/opt/llvm@13/bin/clang-cpp Running ['/usr/local/opt/llvm@13/bin/clang-cpp', '--version'] Homebrew clang version 13.0.1 Target: x86_64-apple-darwin22.6.0 Thread model: posix InstalledDir: /usr/local/opt/llvm@13/bin Inferred XLD as /usr/local/opt/llvm@13/bin/ld.lld Running ['/usr/local/opt/llvm@13/bin/ld.lld', '--version'] Homebrew LLD 13.0.1 (compatible with GNU linkers) Source bmake version: 20230909 Installed bmake version: 20230909 Configure args: --with-default-sys-path=.../share/mk:/Users/.../freebsd-obj/bmake-install/share/mk --with-machine=amd64 --without-filemon --prefix=/Users/.../freebsd-obj/bmake-install Last configure args: --with-default-sys-path=.../share/mk:/Users/.../freebsd-obj/bmake-install/share/mk --with-machine=amd64 --without-filemon --prefix=/Users/.../freebsd-obj/bmake-install Adding -DWITH_AUTO_OBJ Running `env CC=/usr/bin/cc CXX=/usr/bin/c++ CPP=/usr/bin/cpp XCC=/usr/local/opt/llvm@13/bin/clang XCXX=/usr/local/opt/llvm@13/bin/clang++ XCPP=/usr/local/opt/llvm@13/bin/clang-cpp XLD=/usr/local/opt/llvm@13/bin/ld.lld /Users/.../freebsd-obj/bmake-install/bin/bmake TARGET=riscv TARGET_ARCH=riscv64 kernel-toolchain -s -DWITH_DISK_IMAGE_TOOLS_BOOTSTRAP -DWITH_AUTO_OBJ -DWITHOUT_CLEAN` >>> Deleting stale files in build tree... 6.29 real 4.95 user 1.61 sys -------------------------------------------------------------- >>> Rebuilding the temporary build tree -------------------------------------------------------------- Linking host tools into /Users/.../freebsd-obj/Users/.../freebsd-src/riscv.riscv64/tmp/legacy/bin -------------------------------------------------------------- >>> stage 1.1: legacy release compatibility shims -------------------------------------------------------------- ===> tools/build (obj,includes,all,install) real 0m1.938s user 0m0.624s sys 0m0.697s -------------------------------------------------------------- >>> stage 1.2: bootstrap tools -------------------------------------------------------------- >>> Deleting stale dependencies... real 0m0.385s user 0m0.132s sys 0m0.211s ===> lib/clang/libllvmminimal (obj,all,install) ===> usr.bin/clang/llvm-tblgen (obj,all,install) bmake[3]: /Users/.../freebsd-obj/Users/.../freebsd-src/riscv.riscv64/tmp/obj-tools/usr.bin/clang/llvm-tblgen/.depend, 1: ignoring stale .depend for /Users/.../freebsd-obj//Users/.../freebsd-src/riscv.riscv64/tmp/legacy/usr/lib/libz.a bmake[3]: /Users/.../freebsd-obj/Users/.../freebsd-src/riscv.riscv64/tmp/obj-tools/usr.bin/clang/llvm-tblgen/.depend, 1: ignoring stale .depend for /usr/lib/libprivatezstd.a ld: warning: directory not found for option '-L/Users/.../freebsd-obj/Users/.../freebsd-src/riscv.riscv64/tmp/obj-tools/lib/libthr' ld: warning: directory not found for option '-L/Users/.../freebsd-obj/Users/.../freebsd-src/riscv.riscv64/tmp/obj-tools/lib/libz' ld: warning: directory not found for option '-L/Users/.../freebsd-obj/Users/.../freebsd-src/riscv.riscv64/tmp/obj-tools/lib/libthr' ld: library not found for -lprivatezstd clang: error: linker command failed with exit code 1 (use -v to see invocation) *** Error code 1 Stop. bmake[3]: stopped in /Users/.../freebsd-src/usr.bin/clang/llvm-tblgen *** Error code 1 Stop. bmake[2]: stopped in /Users/.../freebsd-src *** Error code 1 Stop. bmake[1]: stopped in /Users/.../freebsd-src *** Error code 1 Stop. bmake: stopped in /Users/.../freebsd-src I get the same failure if I just try to buildworld instead of build kernel-toolchain. Above I'm trying to cross-build riscv64, but the same things happens when I try other platforms as well (e.g., aarch64, which specifically is built in CI). I figure relatively few people have actually debugged a cross-build from macos, but has anyone seen similar build errors about privatezstd before, or other llvm-tblgen build failures? Thanks for any pointers, Colin --cc114965f19149b8a12b00c0d4ca10ba Content-Type: text/html Content-Transfer-Encoding: quoted-printable <!DOCTYPE html><html><head><title></title><style type=3D"text/css">p.Mso= Normal,p.MsoNoSpacing{margin:0}</style></head><body><div>This is a longs= hot but: I'm trying to get cross-building working from my mac so I don't= have to lug 2 laptops around every day to make any progress on some dri= ver work, but so far haven't had too much luck. This is (to my ple= asant surprise!) supported (<a href=3D"https://docs.freebsd.org/en/books= /handbook/cutting-edge/#building-on-non-freebsd-hosts">https://docs.free= bsd.org/en/books/handbook/cutting-edge/#building-on-non-freebsd-hosts</a= >) and as Ed pointed out on Mastodon, even runs successfully in CI (<a h= ref=3D"https://github.com/freebsd/freebsd-src/blob/main/.github/workflow= s/cross-bootstrap-tools.yml">https://github.com/freebsd/freebsd-src/blob= /main/.github/workflows/cross-bootstrap-tools.yml</a>). But so far I'm u= nable get this working on my own machine. I was initially trying t= o get this running with my slightly-stale branch where I'm doing some RI= SC-V stuff, but hit the same problem building a recent commit from HEAD = that does cross-build from macos in CI, as evidenced here: <a href=3D"ht= tps://github.com/freebsd/freebsd-src/actions/runs/6801518714/job/1849256= 6834">https://github.com/freebsd/freebsd-src/actions/runs/6801518714/job= /18492566834</a><br></div><div><br></div><div>I've installed the same de= pendencies (same version of LLVM) specified in the github action file, a= nd am building with the same commands (with different local directories,= of course). I appear to be one macos release ahead of the version= used in CI, though it doesn't seem like that should affect the failure = I'm encountering. The first command,<br></div><div><br></div><div>= MAKEOBJDIRPREFIX=3D~/freebsd-obj/ tools/build/make.py= --debug --cross-bindir=3D/usr/local/opt/llvm@13/bin TARGET=3Driscv TARG= ET_ARCH=3Driscv64 -n<br></div><div><br></div><div>seems to work fine. Bu= t the second, a pre-requisite to building the kernel, fails:<br></div><d= iv><br></div><div>MAKEOBJDIRPREFIX=3D~/freebsd-obj/ tools/build/make.py = --debug --cross-bindir=3D/usr/local/opt/llvm@13/bin TARGET=3Driscv TARGE= T_ARCH=3Driscv64 kernel-toolchain -s -DWITH_DISK_IMAGE_TOOLS_BOOTSTRAP<b= r></div><div>Inferred CC as /usr/bin/cc<br></div><div>Running ['/usr/bin= /cc', '--version']<br></div><div>Apple clang version 14.0.3 (clang-1403.= 0.22.14.1)<br></div><div>Target: x86_64-apple-darwin22.6.0<br></div><div= >Thread model: posix<br></div><div>InstalledDir: /Library/Developer/Comm= andLineTools/usr/bin<br></div><div>Inferred CXX as /usr/bin/c++<br></div= ><div>Running ['/usr/bin/c++', '--version']<br></div><div>Apple clang ve= rsion 14.0.3 (clang-1403.0.22.14.1)<br></div><div>Target: x86_64-apple-d= arwin22.6.0<br></div><div>Thread model: posix<br></div><div>InstalledDir= : /Library/Developer/CommandLineTools/usr/bin<br></div><div>Inferred CPP= as /usr/bin/cpp<br></div><div>Running ['/usr/bin/cpp', '--version']<br>= </div><div>Apple clang version 14.0.3 (clang-1403.0.22.14.1)<br></div><d= iv>Target: x86_64-apple-darwin22.6.0<br></div><div>Thread model: posix<b= r></div><div>InstalledDir: /Library/Developer/CommandLineTools/usr/bin<b= r></div><div>Inferred XCC as /usr/local/opt/llvm@13/bin/clang<br></div><= div>Running ['/usr/local/opt/llvm@13/bin/clang', '--version']<br></div><= div>Homebrew clang version 13.0.1<br></div><div>Target: x86_64-apple-dar= win22.6.0<br></div><div>Thread model: posix<br></div><div>InstalledDir: = /usr/local/opt/llvm@13/bin<br></div><div>Inferred XCXX as /usr/local/opt= /llvm@13/bin/clang++<br></div><div>Running ['/usr/local/opt/llvm@13/bin/= clang++', '--version']<br></div><div>Homebrew clang version 13.0.1<br></= div><div>Target: x86_64-apple-darwin22.6.0<br></div><div>Thread model: p= osix<br></div><div>InstalledDir: /usr/local/opt/llvm@13/bin<br></div><di= v>Inferred XCPP as /usr/local/opt/llvm@13/bin/clang-cpp<br></div><div>Ru= nning ['/usr/local/opt/llvm@13/bin/clang-cpp', '--version']<br></div><di= v>Homebrew clang version 13.0.1<br></div><div>Target: x86_64-apple-darwi= n22.6.0<br></div><div>Thread model: posix<br></div><div>InstalledDir: /u= sr/local/opt/llvm@13/bin<br></div><div>Inferred XLD as /usr/local/opt/ll= vm@13/bin/ld.lld<br></div><div>Running ['/usr/local/opt/llvm@13/bin/ld.l= ld', '--version']<br></div><div>Homebrew LLD 13.0.1 (compatible with GNU= linkers)<br></div><div>Source bmake version: 20230909<br></div><div>Ins= talled bmake version: 20230909<br></div><div>Configure args: --with-defa= ult-sys-path=3D.../share/mk:/Users/.../freebsd-obj/bmake-install/share/m= k --with-machine=3Damd64 --without-filemon --prefix=3D/Users/.../freebsd= -obj/bmake-install<br></div><div>Last configure args: --with-default-sys= -path=3D.../share/mk:/Users/.../freebsd-obj/bmake-install/share/mk --wit= h-machine=3Damd64 --without-filemon --prefix=3D/Users/.../freebsd-obj/bm= ake-install<br></div><div>Adding -DWITH_AUTO_OBJ<br></div><div>Running `= env CC=3D/usr/bin/cc CXX=3D/usr/bin/c++ CPP=3D/usr/bin/cpp XCC=3D/usr/lo= cal/opt/llvm@13/bin/clang XCXX=3D/usr/local/opt/llvm@13/bin/clang++ XCPP= =3D/usr/local/opt/llvm@13/bin/clang-cpp XLD=3D/usr/local/opt/llvm@13/bin= /ld.lld /Users/.../freebsd-obj/bmake-install/bin/bmake TARGET=3Driscv TA= RGET_ARCH=3Driscv64 kernel-toolchain -s -DWITH_DISK_IMAGE_TOOLS_BOOTSTRA= P -DWITH_AUTO_OBJ -DWITHOUT_CLEAN`<br></div><div>>>> Deleting s= tale files in build tree...<br></div><div> = 6.29 real 4= .95 user 1.61 sys<br></d= iv><div><br></div><div>-------------------------------------------------= -------------<br></div><div>>>> Rebuilding the temporary build = tree<br></div><div>-----------------------------------------------------= ---------<br></div><div>Linking host tools into /Users/.../freebsd-obj/U= sers/.../freebsd-src/riscv.riscv64/tmp/legacy/bin<br></div><div><br></di= v><div>--------------------------------------------------------------<br= ></div><div>>>> stage 1.1: legacy release compatibility shims<b= r></div><div>-----------------------------------------------------------= ---<br></div><div>=3D=3D=3D> tools/build (obj,includes,all,install)<b= r></div><div><br></div><div>real 0m1.938s<br></div><div>user 0m0.624s<br= ></div><div>sys 0m0.697s<br></div><div><br></div><div>------------------= --------------------------------------------<br></div><div>>>> = stage 1.2: bootstrap tools<br></div><div>-------------------------------= -------------------------------<br></div><div>>>> Deleting stal= e dependencies...<br></div><div><br></div><div>real 0m0.385s<br></div><d= iv>user 0m0.132s<br></div><div>sys 0m0.211s<br></div><div>=3D=3D=3D> = lib/clang/libllvmminimal (obj,all,install)<br></div><div>=3D=3D=3D> <= a href=3D"http://usr.bin/clang/llvm-tblgen">usr.bin/clang/llvm-tblgen</a= > (obj,all,install)<br></div><div>bmake[3]: /Users/.../freebsd-obj/Users= /.../freebsd-src/riscv.riscv64/tmp/obj-tools/<a href=3D"http://usr.bin/c= lang/llvm-tblgen/.depend">usr.bin/clang/llvm-tblgen/.depend</a>, 1: igno= ring stale .depend for /Users/.../freebsd-obj//Users/.../freebsd-src/ris= cv.riscv64/tmp/legacy/usr/lib/libz.a<br></div><div>bmake[3]: /Users/.../= freebsd-obj/Users/.../freebsd-src/riscv.riscv64/tmp/obj-tools/<a href=3D= "http://usr.bin/clang/llvm-tblgen/.depend">usr.bin/clang/llvm-tblgen/.de= pend</a>, 1: ignoring stale .depend for /usr/lib/libprivatezstd.a<br></d= iv><div>ld: warning: directory not found for option '-L/Users/.../freebs= d-obj/Users/.../freebsd-src/riscv.riscv64/tmp/obj-tools/lib/libthr'<br><= /div><div>ld: warning: directory not found for option '-L/Users/.../free= bsd-obj/Users/.../freebsd-src/riscv.riscv64/tmp/obj-tools/lib/libz'<br><= /div><div>ld: warning: directory not found for option '-L/Users/.../free= bsd-obj/Users/.../freebsd-src/riscv.riscv64/tmp/obj-tools/lib/libthr'<br= ></div><div>ld: library not found for -lprivatezstd<br></div><div>clang:= error: linker command failed with exit code 1 (use -v to see invocation= )<br></div><div>*** Error code 1<br></div><div><br></div><div>Stop.<br><= /div><div>bmake[3]: stopped in /Users/.../freebsd-src/<a href=3D"http://= usr.bin/clang/llvm-tblgen">usr.bin/clang/llvm-tblgen</a><br></div><div>*= ** Error code 1<br></div><div><br></div><div>Stop.<br></div><div>bmake[2= ]: stopped in /Users/.../freebsd-src<br></div><div>*** Error code 1<br><= /div><div><br></div><div>Stop.<br></div><div>bmake[1]: stopped in /Users= /.../freebsd-src<br></div><div>*** Error code 1<br></div><div><br></div>= <div>Stop.<br></div><div>bmake: stopped in /Users/.../freebsd-src<br></d= iv><div><br></div><div><br></div><div>I get the same failure if I just t= ry to buildworld instead of build kernel-toolchain. Above I'm trying to = cross-build riscv64, but the same things happens when I try other platfo= rms as well (e.g., aarch64, which specifically is built in CI).<br></div= ><div><br></div><div>I figure relatively few people have actually debugg= ed a cross-build from macos, but has anyone seen similar build errors ab= out privatezstd before, or other llvm-tblgen build failures?<br></div><d= iv><br></div><div>Thanks for any pointers,<br></div><div>Colin<br></div>= </body></html> --cc114965f19149b8a12b00c0d4ca10ba--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?f42da895-b8e3-49a8-826a-d448103b047b>