Date: Wed, 08 Nov 2023 14:54:33 -0500 From: "Colin S. Gordon" <csgordon@fastmail.com> To: freebsd-hackers@FreeBSD.org Subject: Help cross-building kernel from macos Message-ID: <f42da895-b8e3-49a8-826a-d448103b047b@app.fastmail.com>
next in thread | raw e-mail | index | archive | help
[-- Attachment #1 --] This is a longshot but: I'm trying to get cross-building working from my mac so I don't have to lug 2 laptops around every day to make any progress on some driver work, but so far haven't had too much luck. This is (to my pleasant surprise!) supported (https://docs.freebsd.org/en/books/handbook/cutting-edge/#building-on-non-freebsd-hosts) and as Ed pointed out on Mastodon, even runs successfully in CI (https://github.com/freebsd/freebsd-src/blob/main/.github/workflows/cross-bootstrap-tools.yml). But so far I'm unable get this working on my own machine. I was initially trying to get this running with my slightly-stale branch where I'm doing some RISC-V stuff, but hit the same problem building a recent commit from HEAD that does cross-build from macos in CI, as evidenced here: https://github.com/freebsd/freebsd-src/actions/runs/6801518714/job/18492566834 I've installed the same dependencies (same version of LLVM) specified in the github action file, and am building with the same commands (with different local directories, of course). I appear to be one macos release ahead of the version used in CI, though it doesn't seem like that should affect the failure I'm encountering. The first command, MAKEOBJDIRPREFIX=~/freebsd-obj/ tools/build/make.py --debug --cross-bindir=/usr/local/opt/llvm@13/bin TARGET=riscv TARGET_ARCH=riscv64 -n seems to work fine. But the second, a pre-requisite to building the kernel, fails: MAKEOBJDIRPREFIX=~/freebsd-obj/ tools/build/make.py --debug --cross-bindir=/usr/local/opt/llvm@13/bin TARGET=riscv TARGET_ARCH=riscv64 kernel-toolchain -s -DWITH_DISK_IMAGE_TOOLS_BOOTSTRAP Inferred CC as /usr/bin/cc Running ['/usr/bin/cc', '--version'] Apple clang version 14.0.3 (clang-1403.0.22.14.1) Target: x86_64-apple-darwin22.6.0 Thread model: posix InstalledDir: /Library/Developer/CommandLineTools/usr/bin Inferred CXX as /usr/bin/c++ Running ['/usr/bin/c++', '--version'] Apple clang version 14.0.3 (clang-1403.0.22.14.1) Target: x86_64-apple-darwin22.6.0 Thread model: posix InstalledDir: /Library/Developer/CommandLineTools/usr/bin Inferred CPP as /usr/bin/cpp Running ['/usr/bin/cpp', '--version'] Apple clang version 14.0.3 (clang-1403.0.22.14.1) Target: x86_64-apple-darwin22.6.0 Thread model: posix InstalledDir: /Library/Developer/CommandLineTools/usr/bin Inferred XCC as /usr/local/opt/llvm@13/bin/clang Running ['/usr/local/opt/llvm@13/bin/clang', '--version'] Homebrew clang version 13.0.1 Target: x86_64-apple-darwin22.6.0 Thread model: posix InstalledDir: /usr/local/opt/llvm@13/bin Inferred XCXX as /usr/local/opt/llvm@13/bin/clang++ Running ['/usr/local/opt/llvm@13/bin/clang++', '--version'] Homebrew clang version 13.0.1 Target: x86_64-apple-darwin22.6.0 Thread model: posix InstalledDir: /usr/local/opt/llvm@13/bin Inferred XCPP as /usr/local/opt/llvm@13/bin/clang-cpp Running ['/usr/local/opt/llvm@13/bin/clang-cpp', '--version'] Homebrew clang version 13.0.1 Target: x86_64-apple-darwin22.6.0 Thread model: posix InstalledDir: /usr/local/opt/llvm@13/bin Inferred XLD as /usr/local/opt/llvm@13/bin/ld.lld Running ['/usr/local/opt/llvm@13/bin/ld.lld', '--version'] Homebrew LLD 13.0.1 (compatible with GNU linkers) Source bmake version: 20230909 Installed bmake version: 20230909 Configure args: --with-default-sys-path=.../share/mk:/Users/.../freebsd-obj/bmake-install/share/mk --with-machine=amd64 --without-filemon --prefix=/Users/.../freebsd-obj/bmake-install Last configure args: --with-default-sys-path=.../share/mk:/Users/.../freebsd-obj/bmake-install/share/mk --with-machine=amd64 --without-filemon --prefix=/Users/.../freebsd-obj/bmake-install Adding -DWITH_AUTO_OBJ Running `env CC=/usr/bin/cc CXX=/usr/bin/c++ CPP=/usr/bin/cpp XCC=/usr/local/opt/llvm@13/bin/clang XCXX=/usr/local/opt/llvm@13/bin/clang++ XCPP=/usr/local/opt/llvm@13/bin/clang-cpp XLD=/usr/local/opt/llvm@13/bin/ld.lld /Users/.../freebsd-obj/bmake-install/bin/bmake TARGET=riscv TARGET_ARCH=riscv64 kernel-toolchain -s -DWITH_DISK_IMAGE_TOOLS_BOOTSTRAP -DWITH_AUTO_OBJ -DWITHOUT_CLEAN` >>> Deleting stale files in build tree... 6.29 real 4.95 user 1.61 sys -------------------------------------------------------------- >>> Rebuilding the temporary build tree -------------------------------------------------------------- Linking host tools into /Users/.../freebsd-obj/Users/.../freebsd-src/riscv.riscv64/tmp/legacy/bin -------------------------------------------------------------- >>> stage 1.1: legacy release compatibility shims -------------------------------------------------------------- ===> tools/build (obj,includes,all,install) real 0m1.938s user 0m0.624s sys 0m0.697s -------------------------------------------------------------- >>> stage 1.2: bootstrap tools -------------------------------------------------------------- >>> Deleting stale dependencies... real 0m0.385s user 0m0.132s sys 0m0.211s ===> lib/clang/libllvmminimal (obj,all,install) ===> usr.bin/clang/llvm-tblgen (obj,all,install) bmake[3]: /Users/.../freebsd-obj/Users/.../freebsd-src/riscv.riscv64/tmp/obj-tools/usr.bin/clang/llvm-tblgen/.depend, 1: ignoring stale .depend for /Users/.../freebsd-obj//Users/.../freebsd-src/riscv.riscv64/tmp/legacy/usr/lib/libz.a bmake[3]: /Users/.../freebsd-obj/Users/.../freebsd-src/riscv.riscv64/tmp/obj-tools/usr.bin/clang/llvm-tblgen/.depend, 1: ignoring stale .depend for /usr/lib/libprivatezstd.a ld: warning: directory not found for option '-L/Users/.../freebsd-obj/Users/.../freebsd-src/riscv.riscv64/tmp/obj-tools/lib/libthr' ld: warning: directory not found for option '-L/Users/.../freebsd-obj/Users/.../freebsd-src/riscv.riscv64/tmp/obj-tools/lib/libz' ld: warning: directory not found for option '-L/Users/.../freebsd-obj/Users/.../freebsd-src/riscv.riscv64/tmp/obj-tools/lib/libthr' ld: library not found for -lprivatezstd clang: error: linker command failed with exit code 1 (use -v to see invocation) *** Error code 1 Stop. bmake[3]: stopped in /Users/.../freebsd-src/usr.bin/clang/llvm-tblgen *** Error code 1 Stop. bmake[2]: stopped in /Users/.../freebsd-src *** Error code 1 Stop. bmake[1]: stopped in /Users/.../freebsd-src *** Error code 1 Stop. bmake: stopped in /Users/.../freebsd-src I get the same failure if I just try to buildworld instead of build kernel-toolchain. Above I'm trying to cross-build riscv64, but the same things happens when I try other platforms as well (e.g., aarch64, which specifically is built in CI). I figure relatively few people have actually debugged a cross-build from macos, but has anyone seen similar build errors about privatezstd before, or other llvm-tblgen build failures? Thanks for any pointers, Colin [-- Attachment #2 --] <!DOCTYPE html><html><head><title></title><style type="text/css">p.MsoNormal,p.MsoNoSpacing{margin:0}</style></head><body><div>This is a longshot but: I'm trying to get cross-building working from my mac so I don't have to lug 2 laptops around every day to make any progress on some driver work, but so far haven't had too much luck. This is (to my pleasant surprise!) supported (<a href="https://docs.freebsd.org/en/books/handbook/cutting-edge/#building-on-non-freebsd-hosts">https://docs.freebsd.org/en/books/handbook/cutting-edge/#building-on-non-freebsd-hosts</a>) and as Ed pointed out on Mastodon, even runs successfully in CI (<a href="https://github.com/freebsd/freebsd-src/blob/main/.github/workflows/cross-bootstrap-tools.yml">https://github.com/freebsd/freebsd-src/blob/main/.github/workflows/cross-bootstrap-tools.yml</a>). But so far I'm unable get this working on my own machine. I was initially trying to get this running with my slightly-stale branch where I'm doing some RISC-V stuff, but hit the same problem building a recent commit from HEAD that does cross-build from macos in CI, as evidenced here: <a href="https://github.com/freebsd/freebsd-src/actions/runs/6801518714/job/18492566834">https://github.com/freebsd/freebsd-src/actions/runs/6801518714/job/18492566834</a><br></div><div><br></div><div>I've installed the same dependencies (same version of LLVM) specified in the github action file, and am building with the same commands (with different local directories, of course). I appear to be one macos release ahead of the version used in CI, though it doesn't seem like that should affect the failure I'm encountering. The first command,<br></div><div><br></div><div> MAKEOBJDIRPREFIX=~/freebsd-obj/ tools/build/make.py --debug --cross-bindir=/usr/local/opt/llvm@13/bin TARGET=riscv TARGET_ARCH=riscv64 -n<br></div><div><br></div><div>seems to work fine. But the second, a pre-requisite to building the kernel, fails:<br></div><div><br></div><div>MAKEOBJDIRPREFIX=~/freebsd-obj/ tools/build/make.py --debug --cross-bindir=/usr/local/opt/llvm@13/bin TARGET=riscv TARGET_ARCH=riscv64 kernel-toolchain -s -DWITH_DISK_IMAGE_TOOLS_BOOTSTRAP<br></div><div>Inferred CC as /usr/bin/cc<br></div><div>Running ['/usr/bin/cc', '--version']<br></div><div>Apple clang version 14.0.3 (clang-1403.0.22.14.1)<br></div><div>Target: x86_64-apple-darwin22.6.0<br></div><div>Thread model: posix<br></div><div>InstalledDir: /Library/Developer/CommandLineTools/usr/bin<br></div><div>Inferred CXX as /usr/bin/c++<br></div><div>Running ['/usr/bin/c++', '--version']<br></div><div>Apple clang version 14.0.3 (clang-1403.0.22.14.1)<br></div><div>Target: x86_64-apple-darwin22.6.0<br></div><div>Thread model: posix<br></div><div>InstalledDir: /Library/Developer/CommandLineTools/usr/bin<br></div><div>Inferred CPP as /usr/bin/cpp<br></div><div>Running ['/usr/bin/cpp', '--version']<br></div><div>Apple clang version 14.0.3 (clang-1403.0.22.14.1)<br></div><div>Target: x86_64-apple-darwin22.6.0<br></div><div>Thread model: posix<br></div><div>InstalledDir: /Library/Developer/CommandLineTools/usr/bin<br></div><div>Inferred XCC as /usr/local/opt/llvm@13/bin/clang<br></div><div>Running ['/usr/local/opt/llvm@13/bin/clang', '--version']<br></div><div>Homebrew clang version 13.0.1<br></div><div>Target: x86_64-apple-darwin22.6.0<br></div><div>Thread model: posix<br></div><div>InstalledDir: /usr/local/opt/llvm@13/bin<br></div><div>Inferred XCXX as /usr/local/opt/llvm@13/bin/clang++<br></div><div>Running ['/usr/local/opt/llvm@13/bin/clang++', '--version']<br></div><div>Homebrew clang version 13.0.1<br></div><div>Target: x86_64-apple-darwin22.6.0<br></div><div>Thread model: posix<br></div><div>InstalledDir: /usr/local/opt/llvm@13/bin<br></div><div>Inferred XCPP as /usr/local/opt/llvm@13/bin/clang-cpp<br></div><div>Running ['/usr/local/opt/llvm@13/bin/clang-cpp', '--version']<br></div><div>Homebrew clang version 13.0.1<br></div><div>Target: x86_64-apple-darwin22.6.0<br></div><div>Thread model: posix<br></div><div>InstalledDir: /usr/local/opt/llvm@13/bin<br></div><div>Inferred XLD as /usr/local/opt/llvm@13/bin/ld.lld<br></div><div>Running ['/usr/local/opt/llvm@13/bin/ld.lld', '--version']<br></div><div>Homebrew LLD 13.0.1 (compatible with GNU linkers)<br></div><div>Source bmake version: 20230909<br></div><div>Installed bmake version: 20230909<br></div><div>Configure args: --with-default-sys-path=.../share/mk:/Users/.../freebsd-obj/bmake-install/share/mk --with-machine=amd64 --without-filemon --prefix=/Users/.../freebsd-obj/bmake-install<br></div><div>Last configure args: --with-default-sys-path=.../share/mk:/Users/.../freebsd-obj/bmake-install/share/mk --with-machine=amd64 --without-filemon --prefix=/Users/.../freebsd-obj/bmake-install<br></div><div>Adding -DWITH_AUTO_OBJ<br></div><div>Running `env CC=/usr/bin/cc CXX=/usr/bin/c++ CPP=/usr/bin/cpp XCC=/usr/local/opt/llvm@13/bin/clang XCXX=/usr/local/opt/llvm@13/bin/clang++ XCPP=/usr/local/opt/llvm@13/bin/clang-cpp XLD=/usr/local/opt/llvm@13/bin/ld.lld /Users/.../freebsd-obj/bmake-install/bin/bmake TARGET=riscv TARGET_ARCH=riscv64 kernel-toolchain -s -DWITH_DISK_IMAGE_TOOLS_BOOTSTRAP -DWITH_AUTO_OBJ -DWITHOUT_CLEAN`<br></div><div>>>> Deleting stale files in build tree...<br></div><div> 6.29 real 4.95 user 1.61 sys<br></div><div><br></div><div>--------------------------------------------------------------<br></div><div>>>> Rebuilding the temporary build tree<br></div><div>--------------------------------------------------------------<br></div><div>Linking host tools into /Users/.../freebsd-obj/Users/.../freebsd-src/riscv.riscv64/tmp/legacy/bin<br></div><div><br></div><div>--------------------------------------------------------------<br></div><div>>>> stage 1.1: legacy release compatibility shims<br></div><div>--------------------------------------------------------------<br></div><div>===> tools/build (obj,includes,all,install)<br></div><div><br></div><div>real 0m1.938s<br></div><div>user 0m0.624s<br></div><div>sys 0m0.697s<br></div><div><br></div><div>--------------------------------------------------------------<br></div><div>>>> stage 1.2: bootstrap tools<br></div><div>--------------------------------------------------------------<br></div><div>>>> Deleting stale dependencies...<br></div><div><br></div><div>real 0m0.385s<br></div><div>user 0m0.132s<br></div><div>sys 0m0.211s<br></div><div>===> lib/clang/libllvmminimal (obj,all,install)<br></div><div>===> <a href="http://usr.bin/clang/llvm-tblgen">usr.bin/clang/llvm-tblgen</a>; (obj,all,install)<br></div><div>bmake[3]: /Users/.../freebsd-obj/Users/.../freebsd-src/riscv.riscv64/tmp/obj-tools/<a href="http://usr.bin/clang/llvm-tblgen/.depend">usr.bin/clang/llvm-tblgen/.depend</a>, 1: ignoring stale .depend for /Users/.../freebsd-obj//Users/.../freebsd-src/riscv.riscv64/tmp/legacy/usr/lib/libz.a<br></div><div>bmake[3]: /Users/.../freebsd-obj/Users/.../freebsd-src/riscv.riscv64/tmp/obj-tools/<a href="http://usr.bin/clang/llvm-tblgen/.depend">usr.bin/clang/llvm-tblgen/.depend</a>, 1: ignoring stale .depend for /usr/lib/libprivatezstd.a<br></div><div>ld: warning: directory not found for option '-L/Users/.../freebsd-obj/Users/.../freebsd-src/riscv.riscv64/tmp/obj-tools/lib/libthr'<br></div><div>ld: warning: directory not found for option '-L/Users/.../freebsd-obj/Users/.../freebsd-src/riscv.riscv64/tmp/obj-tools/lib/libz'<br></div><div>ld: warning: directory not found for option '-L/Users/.../freebsd-obj/Users/.../freebsd-src/riscv.riscv64/tmp/obj-tools/lib/libthr'<br></div><div>ld: library not found for -lprivatezstd<br></div><div>clang: error: linker command failed with exit code 1 (use -v to see invocation)<br></div><div>*** Error code 1<br></div><div><br></div><div>Stop.<br></div><div>bmake[3]: stopped in /Users/.../freebsd-src/<a href="http://usr.bin/clang/llvm-tblgen">usr.bin/clang/llvm-tblgen</a><br></div><div>*** Error code 1<br></div><div><br></div><div>Stop.<br></div><div>bmake[2]: stopped in /Users/.../freebsd-src<br></div><div>*** Error code 1<br></div><div><br></div><div>Stop.<br></div><div>bmake[1]: stopped in /Users/.../freebsd-src<br></div><div>*** Error code 1<br></div><div><br></div><div>Stop.<br></div><div>bmake: stopped in /Users/.../freebsd-src<br></div><div><br></div><div><br></div><div>I get the same failure if I just try to buildworld instead of build kernel-toolchain. Above I'm trying to cross-build riscv64, but the same things happens when I try other platforms as well (e.g., aarch64, which specifically is built in CI).<br></div><div><br></div><div>I figure relatively few people have actually debugged a cross-build from macos, but has anyone seen similar build errors about privatezstd before, or other llvm-tblgen build failures?<br></div><div><br></div><div>Thanks for any pointers,<br></div><div>Colin<br></div></body></html>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?f42da895-b8e3-49a8-826a-d448103b047b>