From owner-freebsd-arch@FreeBSD.ORG Mon Jul 21 20:06:07 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9C08770C; Mon, 21 Jul 2014 20:06:07 +0000 (UTC) Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "tensor.andric.com", Issuer "CAcert Class 3 Root" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 9DB532E1D; Mon, 21 Jul 2014 20:06:06 +0000 (UTC) Received: from [IPv6:2001:7b8:3a7::a5aa:5821:70ec:fa81] (unknown [IPv6:2001:7b8:3a7:0:a5aa:5821:70ec:fa81]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id D84F25C44; Mon, 21 Jul 2014 22:06:01 +0200 (CEST) Content-Type: multipart/signed; boundary="Apple-Mail=_C32C06B5-EAAB-4B90-B807-E0690CCC4FBF"; protocol="application/pgp-signature"; micalg=pgp-sha1 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Total confusion over toolchain/xdev behavior From: Dimitry Andric In-Reply-To: <1405706147.19254.17.camel@bruno> Date: Mon, 21 Jul 2014 22:05:55 +0200 Message-Id: References: <1404688077.1059.115.camel@bruno> <1405706147.19254.17.camel@bruno> To: Sean Bruno X-Mailer: Apple Mail (2.1878.6) Cc: Baptiste Daroussin , freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jul 2014 20:06:07 -0000 --Apple-Mail=_C32C06B5-EAAB-4B90-B807-E0690CCC4FBF Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii On 18 Jul 2014, at 19:55, Sean Bruno wrote: > On Sun, 2014-07-06 at 16:07 -0700, Sean Bruno wrote: >> Objective: install an xcompile toolchain into a jail for use by >> poudriere during arm/mips/sparc/power ports pkgs builds. The build >> should be possible from a non-root user. >>=20 >> As far as I can tell, the xdev target is completely busted for >> non-clang >> arch's right now as it tries to build clang no matter what I do. Its >> missing some pretty key documentation to making it work correctly, so >> a >> lot of my attempts have been "guess and check" with verbose make. >>=20 >>=20 >=20 > Quite a bit of success with one blocking failure. Thanks to Warner, > Baptiste and Dimitry for plugging along through some of my ranting (on > bcc here). >=20 > The xdev target can be used to produce a compiler toolchain that can = be > used to build ports. However, final linking seems to fail and makes = it > impossible to use for building many, many ports. >=20 > Regardless of the cross compile bits, simply using the xdev target for > building ports natively on amd64 for amd64 manifests this issue, this > leads me to believe that xdev is *not* building the tool chain > correctly. There is no chroot/jail/emualtion involved in this test > case. >=20 > My test case: >=20 > 1. build amd64 xdev tool chain: > make -s -j4 xdev XDEV=3Damd64 XDEV_ARCH=3Damd64 For me, this failed due to a problem with xdev-install trying to install include files into /usr/amd64-freebsd/usr/include/atf-c, which did not exist. I had to add the following part to Makefile.inc1 to make it = work: Index: Makefile.inc1 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- Makefile.inc1 (revision 268920) +++ Makefile.inc1 (working copy) @@ -1942,6 +1942,10 @@ -p ${XDDESTDIR}/usr >/dev/null mtree -deU -f ${.CURDIR}/etc/mtree/BSD.include.dist \ -p ${XDDESTDIR}/usr/include >/dev/null +.if ${MK_TESTS} !=3D "no" + mtree -deU -f ${.CURDIR}/etc/mtree/BSD.tests.dist \ + -p ${XDDESTDIR}/usr >/dev/null +.endif .ORDER: xdev-build _xi-mtree _xi-cross-tools _xi-includes _xi-libraries xdev-install: xdev-build _xi-mtree _xi-cross-tools _xi-includes = _xi-libraries > 2. modify make.conf to use this toolchain: > CC=3D/usr/amd64-freebsd/usr/bin/cc > CPP=3D/usr/amd64-freebsd/usr/bin/cpp > CXX=3D/usr/amd64-freebsd/usr/bin/cc++ > AS=3D/usr/amd64-freebsd/usr/bin/as > NM=3D/usr/amd64-freebsd/usr/bin/nm > RANLIB=3D/usr/amd64-freebsd/usr/bin/ranlib > LD=3D/usr/amd64-freebsd/usr/bin/ld > OBJCOPY=3D/usr/amd64-freebsd/usr/bin/objcopy > SIZE=3D/usr/amd64-freebsd/usr/bin/llvm-size > STRIPBIN=3D/usr/amd64-freebsd/usr/bin/strip >=20 > 3. attempt to build ports-mgmt/pkg In my case, this would fail much earlier, because it would try to use the native nm and ar, not the ones from /usr/amd64-freebsd/usr/bin. I had to specify X_BUILD_FOR=3Damd64-freebsd in make.conf, and modify bsd.port.mk as follows: Index: Mk/bsd.port.mk =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- Mk/bsd.port.mk (revision 361353) +++ Mk/bsd.port.mk (working copy) @@ -1157,13 +1157,16 @@ .else X_SYSROOT=3D /usr/${X_BUILD_FOR} .endif -CC=3D ${X_SYSROOT}/usr/bin/cc -CXX=3D ${X_SYSROOT}/usr/bin/c++ -PKG_ENV+=3D ABI_FILE=3D${X_SYSROOT}/usr/lib/crt1.o -NM=3D ${X_BUILD_FOR}-nm -STRIP_CMD=3D ${X_BUILD_FOR}-strip -MAKE_ENV+=3D NM=3D${NM} STRIPBIN=3D${X_BUILD_FOR}-strip = PKG_CONFIG_SYSROOT_DIR=3D"${X_SYSROOT}" -CONFIGURE_ENV+=3D PKG_CONFIG_SYSROOT_DIR=3D"${X_SYSROOT}" +.for t in ADDR2LINE AR AS CC CPP CPPFILT GPROF LD NM OBJCOPY OBJDUMP = RANLIB READELF SIZE STRINGS +${t}=3D ${X_SYSROOT}/usr/bin/${t:C/PP/++/:tl} +CONFIGURE_ENV+=3D ${t}=3D"${${t}}" +MAKE_ENV+=3D ${t}=3D"${${t}}" +.endfor +CXX=3D ${X_SYSROOT}/usr/bin/c++ +STRIP_CMD=3D ${X_SYSROOT}/usr/bin/strip +CONFIGURE_ENV+=3D CXX=3D${CXX} STRIPBIN=3D${STRIP_CMD} = PKG_CONFIG_SYSROOT_DIR=3D"${X_SYSROOT}" +MAKE_ENV+=3D CXX=3D${CXX} STRIPBIN=3D${STRIP_CMD} = PKG_CONFIG_SYSROOT_DIR=3D"${X_SYSROOT}" +PKG_ENV+=3D ABI_FILE=3D${X_SYSROOT}/usr/lib/crt1.o # only bmake support the below STRIPBIN=3D ${STRIP_CMD} .export.env STRIPBIN This to ensure that all cross-tools are passed as environment variables to configure and make. > *NOTE* if you add -lbsdxml to CFLAGS this will not happen. Other > packages manfiest similar issues, with different libs. After quite a bit of debugging and instrumenting ld, it turns out this is because the "native" ld has access to a populated hints file (/var/run/ld-elf.so.hints), while the "cross" ld in /usr/amd64-freebsd does not. This can be seen as a bug, or at least inconsistent behavior on the part of ld. The code path followed in each case is subtly different. I'm still investigating what upstream did with it. For now, the way of least resistance would seem to be building ldconfig as part of xdev, and running it post-install. However, it turns out the hints file path is rather hardcoded in ldconfig, so to have that work, it should be made configurable at build time first. That said, the behavior of the "cross" ld seems *more* correct though, in my opinion: our choice (or at least the intention of it) was to stop pulling in DT_NEEDED libraries automatically, for compatibility with newer upstream binutils. E.g. adding just -larchive to a program that uses it should fail with several missing symbols, and you should manually specify the full dependency list: -larchive -lz -bz2 -llzma -lbsdxml -lcrypto This is also what the latest ld 2.24 requires. -Dimitry --Apple-Mail=_C32C06B5-EAAB-4B90-B807-E0690CCC4FBF Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.22 (Darwin) iEYEARECAAYFAlPNcqgACgkQsF6jCi4glqOtjwCfTMz8IXNNRs3cSAZjYmecEc7Z F7gAn0fDCUpIsGjR+rSZ0s4625mgX0ts =Inrj -----END PGP SIGNATURE----- --Apple-Mail=_C32C06B5-EAAB-4B90-B807-E0690CCC4FBF--