Date: Fri, 01 Sep 2017 17:47:18 -0400 From: Naram Qashat <cyberbotx@cyberbotx.com> To: freebsd-ports@FreeBSD.org Cc: bdrewery@FreeBSD.org Subject: Unable to start 11.0-RELEASE aarch64 jail created by poudriere Message-ID: <d6a9eb85a0127b04c0475cc082ac3453@cyberbotx.com>
next in thread | raw e-mail | index | archive | help
I have been attempting to make an 11.0-RELEASE jail for aarch64 to use
with poudriere, with my world source being existing sources in /usr/src,
but I am unable to start the resulting jail. I am currently using
poudriere-devel 3.1.99.20170803 and use UFS for the file system (I do
not use ZFS at all). I have qemu-user-static installed and can verify
that there is an aarch64 entry in 'binmiscctl list'. The following are
the steps I have taken:
1) Made sure that there is no existing jail named local_aarch64 (the
name I was using for the jail) in poudriere.
2) Made sure to delete my /usr/obj/arm64.aarch64 directory.
3) Built the aarch64 world from /usr/src using 'make -j4 buildworld
TARGET_ARCH=aarch64'.
4) Created the poudriere jail using 'poudriere jail -c -J 4 -j
local_aarch64 -a arm64.aarch64 -f none -m src=/usr/src -x'. (I was told
on IRC by RootWyrm that poudriere-devel and the -x flag when creating
the jail as required for aarch64.)
5) Attempting to chroot into the resulting jail using 'chroot
/usr/local/poudriere/jails/local_aarch64 /bin/sh' results in this:
chroot: /bin/sh: No such file or directory
6) Attempting to start the jail with poudriere using 'poudriere jail -s
-j local_aarch64' results in this:
[00:00:00] Cross-building ports for arm64.aarch64 on amd64 requires QEMU
[00:00:00] Creating the reference jail... done
[00:00:36] Mounting system devices for local_aarch64-default
[00:00:36] Mounting ports/packages/distfiles
[00:00:36] Mounting packages from:
[00:00:36] Setting up native-xtools environment in jail... done
[00:00:36] Raising MAX_EXECUTION_TIME and NOHANG_TIME for QEMU
[00:00:36] Copying latest version of the emulator from:
/usr/local/bin/qemu-aarch64-static
[00:00:36] Copying aarch64-binutils ld from
/usr/local/aarch64-freebsd/bin/ld
[00:00:36] Appending to make.conf:
/usr/local/etc/poudriere.d/local_aarch64-make.conf
/etc/resolv.conf ->
/usr/local/poudriere/data/.m/local_aarch64-default/ref/etc/resolv.conf
[00:00:36] Starting jail local_aarch64-default
/etc/rc.subr: /bin/ps: not found
/etc/rc.subr: /bin/ps: not found
eval: /sbin/ldconfig: not found
eval: sysctl: not found
eval: sysctl: not found
eval: sysctl: not found
/usr/ports/Mk/Scripts/ports_env.sh: grep: not found
sh: /usr/bin/uname: not found
make: "/usr/ports/Mk/bsd.port.mk" line 1128: warning: "/usr/bin/uname
-s" returned non-zero status
sh: /usr/bin/uname: not found
make: "/usr/ports/Mk/bsd.port.mk" line 1133: warning: "/usr/bin/uname
-r" returned non-zero status
make: "/usr/ports/Mk/bsd.port.mk" line 1177: UNAME_r () and OSVERSION
(1100122) do not agree on major version number.
jexec: execvp: ldconfig: No such file or directory
7) Attempting to run any executables created for the jail:
# /usr/local/poudriere/jails/local_aarch64/usr/bin/ldd
/usr/local/poudriere/jails/local_aarch64/usr/bin/ldd: Command not found.
# qemu-aarch64-static
/usr/local/poudriere/jails/local_aarch64/usr/bin/ldd
Unable to load interpreter
# /usr/local/poudriere/jails/local_aarch64/rescue/ls /var/empty
/usr/local/poudriere/jails/local_aarch64/rescue/ls: Command not found.
# qemu-aarch64-static /usr/local/poudriere/jails/local_aarch64/rescue/ls
/var/empty
#
(In other words, only running a statically linked executable with
qemu64-aarch64-static works, all other ways do not work. All the files,
when checked with the 'file' command, do say they are built for ARM
aarch64 as opposed to the host's x86-64.)
As verification, this is within the output of 'binmiscctl list':
[...]
name: aarch64
interpreter: /usr/local/bin/qemu-aarch64-static
flags: ENABLED USE_MASK
magic size: 20
magic offset: 0
magic: 0x7f 0x45 0x4c 0x46 0x02 0x01 0x01 0x00 0x00 0x00 0x00 0x00
0x00 0x00 0x00 0x00 0x02 0x00 0xb7 0x00
mask: 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0x00 0xff 0xff 0xff 0xff
0xff 0xff 0xff 0xff 0xfe 0xff 0xff 0xff
[...]
There were no errors in the buildworld nor in the jail creation via
poudriere. Prior to RootWyrm's suggestion of using poudriere-devel and
the -x flag for jail creation, I had been using poudriere 3.1.19 and
creating the jail without the -x flag. In that scenario, I was able to
successfully chroot into the jail's file system (step 5 above) and I was
able to attempt bulk port building (technically step 6 above although I
had not done that with the 3.1.19 scenario because it actually was
building ports for a while), but building ports failed after a while
with execvp errors (one of which was 'jexec: execvp: /usr/bin/limits: No
such file or directory') and then finally resulted in all 4 builders
getting stuck in the clean_pool status before I stopped the jail.
Thanks,
Naram Qashat
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?d6a9eb85a0127b04c0475cc082ac3453>