From owner-freebsd-arch@FreeBSD.ORG Wed Jul 9 06:09:42 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D65254F9; Wed, 9 Jul 2014 06:09:42 +0000 (UTC) Received: from mail-we0-x229.google.com (mail-we0-x229.google.com [IPv6:2a00:1450:400c:c03::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 15158252A; Wed, 9 Jul 2014 06:09:41 +0000 (UTC) Received: by mail-we0-f169.google.com with SMTP id t60so6959890wes.14 for ; Tue, 08 Jul 2014 23:09:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=26SiAmeApzXlUKiTNg1fGQGH506lHyTCR+mNZNPx8Ks=; b=l5mi8LjBF4uk4vSRbbC67iIwhlLH9x9mukF1CA5SsjFsRJO6wQlSRDpvJi+sCmQbo6 nvDDLZGgiGbceFk4GMvitVKHEcyR69lcZ0ganNtPGvCKxGScKOw9vtHktL8M1bWoHI9T KUMndrA7iQCBcYWXPrmB9jH0KztRKGWwNDyqc4FEYRRlkO1U8Es+McdW5joX4vWjyqRg ywYRnYGXRUOg7+WQiVGi9zlg//f8JFsTEjmr4vnEd3qf4BMtmlK551l0KfefeV3oYvXW 2bKPVFpGIR7XawGJHVkf8Nxr6Fe1m1pMWEayGwCAiGdpxkzWteqnByqn2Ar1JqbSQSfp 7qFw== X-Received: by 10.194.83.39 with SMTP id n7mr46132588wjy.58.1404886180156; Tue, 08 Jul 2014 23:09:40 -0700 (PDT) Received: from ivaldir.etoilebsd.net ([2001:41d0:8:db4c::1]) by mx.google.com with ESMTPSA id au7sm44345670wjc.41.2014.07.08.23.09.38 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 08 Jul 2014 23:09:38 -0700 (PDT) Sender: Baptiste Daroussin Date: Wed, 9 Jul 2014 08:09:36 +0200 From: Baptiste Daroussin To: Warner Losh Subject: Re: Total confusion over toolchain/xdev behavior Message-ID: <20140709060936.GB56040@ivaldir.etoilebsd.net> References: <1404688077.1059.115.camel@bruno> <1404766292.65432.43.camel@revolution.hippie.lan> <20B72004-1499-4F99-A7C7-13173C50C7C6@bsdimp.com> <20140707235237.GG97203@ivaldir.etoilebsd.net> <20140709000628.GA56040@ivaldir.etoilebsd.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="8GpibOaaTibBMecb" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: sbruno@FreeBSD.org, Ian Lepore , freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2014 06:09:43 -0000 --8GpibOaaTibBMecb Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Jul 08, 2014 at 09:48:45PM -0600, Warner Losh wrote: >=20 > On Jul 8, 2014, at 6:06 PM, Baptiste Daroussin wrote: >=20 > > On Mon, Jul 07, 2014 at 07:29:01PM -0600, Warner Losh wrote: > >>=20 > >> On Jul 7, 2014, at 5:52 PM, Baptiste Daroussin wrot= e: > >>=20 > >>> On Mon, Jul 07, 2014 at 05:27:25PM -0600, Warner Losh wrote: > >>>>=20 > >>>> On Jul 7, 2014, at 2:51 PM, Ian Lepore wrote: > >>>>=20 > >>>>> On Sun, 2014-07-06 at 16:07 -0700, Sean Bruno wrote: > >>>>>> Objective: install an xcompile toolchain into a jail for use by > >>>>>> poudriere during arm/mips/sparc/power ports pkgs builds. The build > >>>>>> should be possible from a non-root user. > >>>>>>=20 > >>>>>> As far as I can tell, the xdev target is completely busted for non= -clang > >>>>>> arch's right now as it tries to build clang no matter what I do. = Its > >>>>>> missing some pretty key documentation to making it work correctly,= so a > >>>>>> lot of my attempts have been "guess and check" with verbose make. > >>>>>>=20 > >>>>>> ------------------------------------------------------------------= ----- > >>>>>> Attempt #1: > >>>>>> I have been trying non-root xdev builds: > >>>>>> MAKEOBJDIRPREFIX=3D/var/tmp make -s -j8 xdev XDEV=3Dmips XDEV_ARCH= =3Dmips > >>>>>> -- dies because it tries to compile CLANG. > >>>>>> ------------------------------------------------------------------= ----- > >>>>>>=20 > >>>>>> Attempt #2: > >>>>>> Apply a hack from Baptiste that isn't quite right, but at least ge= ts > >>>>>> farther, note the missing variable causing "//usr/mips-freebsd" > >>>>>> http://people.freebsd.org/~sbruno/src.ops.mk.diff > >>>>>>=20 > >>>>>> =3D=3D=3D> gnu/usr.bin/cc/gcov (all) > >>>>>> mtree populating //usr/mips-freebsd > >>>>>> mkdir: //usr/mips-freebsd: Permission denied > >>>>>> *** Error code 1 > >>>>>> ------------------------------------------------------------------= ----- > >>>>>>=20 > >>>>>> Attempt #3: Add XDTP > >>>>>> MAKEOBJDIRPREFIX=3D/var/tmp make -s xdev XDEV=3Dmips XDEV_ARCH=3Dm= ips > >>>>>> XDTP=3D/var/tmp/mips_cc > >>>>>>=20 > >>>>>> Try defining a XDTP=3D/var/tmp/mips_cc with the above patch applie= d, get's > >>>>>> a bit farther but compile failure in locating critical include fil= es. > >>>>>>=20 > >>>>>> =3D=3D=3D> gnu/lib/libstdc++ (obj,depend,all,install) > >>>>>> In file included from /home/sbruno/bsd/fbsd_head/gnu/lib/libstdc > >>>>>> ++/../../../contrib/libstdc++/src/bitmap_allocator.cc:30: > >>>>>> /home/sbruno/bsd/fbsd_head/gnu/lib/libstdc++/../../../contrib/libs= tdc > >>>>>> ++/include/ext/bitmap_allocator.h:37:54: error: cstddef: No such f= ile or > >>>>>> directory > >>>>>> ------------------------------------------------------------------= ----- > >>>>>>=20 > >>>>>> Attempt #4: Add the additional XDDESTDIR > >>>>>> MAKEOBJDIRPREFIX=3D/var/tmp make -s xdev XDEV=3Dmips XDEV_ARCH=3Dm= ips > >>>>>> XDTP=3D/var/tmp/mips_cc XDESTDIR=3D/var/tmp/mips_dst > >>>>>> -- Same results as attempt #3 > >>>>>> ------------------------------------------------------------------= ----- > >>>>>>=20 > >>>>>> Even attempting to do stuff for *clang* enabled architectures bails > >>>>>> because its not respecting prefixes: > >>>>>> MAKEOBJDIRPREFIX=3D/var/tmp make -s -j 8 xdev XDEV=3Darm XDEV_ARCH= =3Darmv6 > >>>>>> -- bails because it tries to: > >>>>>> =3D=3D=3D> usr.bin/clang/tblgen (all) > >>>>>> mtree populating //usr/armv6-freebsd > >>>>>> mtree: etc/ntp: Permission denied > >>>>>> _xi-cross-tools > >>>>>> =3D=3D=3D> xdev gnu/usr.bin/binutils (install) > >>>>>> =3D=3D=3D> gnu/usr.bin/binutils/libiberty (install) > >>>>>> =3D=3D=3D> gnu/usr.bin/binutils/libbfd (install) > >>>>>> =3D=3D=3D> gnu/usr.bin/binutils/libopcodes (install) > >>>>>> =3D=3D=3D> gnu/usr.bin/binutils/libbinutils (install) > >>>>>> =3D=3D=3D> gnu/usr.bin/binutils/addr2line (install) > >>>>>> =3D=3D=3D> gnu/usr.bin/binutils/as (install) > >>>>>> =3D=3D=3D> gnu/usr.bin/binutils/ld (install) > >>>>>> install: //usr/armv6-freebsd/usr/bin/ld: Permission denied > >>>>>> *** Error code 71 > >>>>>>=20 > >>>>>> ------------------------------------------------------------------= ----- > >>>>>> Adding XDTP and XDDESTDIR results in a little more progress but ob= vious > >>>>>> failures to attempt and install things directly into my host syste= m: > >>>>>>=20 > >>>>>> MAKEOBJDIRPREFIX=3D/var/tmp make -s xdev XDEV=3Darm XDEV_ARCH=3Dar= mv6 > >>>>>> XDDESTDIR=3D/var/tmp/arm_cc XDTP=3D/var/tmp/armv6_cc > >>>>>> =3D=3D=3D> secure/lib/libssh (install) > >>>>>> =3D=3D=3D> usr.bin/lex/lib (obj,depend,all,install) > >>>>>> mkdir: ../../../../usr: Permission denied > >>>>>> *** Error code 1 > >>>>>>=20 > >>>>>> Stop. > >>>>>> make[1]: stopped in /home/sbruno/bsd/fbsd_head > >>>>>> *** Error code 1 > >>>>>>=20 > >>>>>> Stop. > >>>>>> make: stopped in /home/sbruno/bsd/fbsd_head > >>>>>=20 > >>>>> It looks to me like the permission part of the problem is being cau= sed > >>>>> by a lack of DESTDIR=3D. Without that, it's trying to install to /= usr and > >>>>> you don't have permission for that. Maybe the confusion is because= the > >>>>> xdev target inherently builds-and-installs, unlike most other targe= ts > >>>>> that separate those two actions. > >>>>=20 > >>>> OK. After some detective work, it looks like libstdc++ needs to be d= one before libsupc++ is done. I=E2=80=99ve added this dependency in r268377= and was able to do a full xdev build with a clean obj dir: > >>>>=20 > >>>> rm -rf $HOME/F $MAKEOBJDIRPREFIX/mips-freebsd > >>>> mkdir $HOME/F > >>>> make xdev DESTDIR=3D$HOME/F XDEV=3Dmips XDEV_ARCH=3Dmips WITHOUT_CL= ANG=3Dt WITHOUT_CLANG_BOOTSTRAP=3Dt WITH_GCC=3Dt WITH_GCC_BOOTSTRAP=3Dt WIT= H_GNUCXX=3Dt -j 20 > >>>=20 > >>> We can avoid most of the above by using a patch like the following: > >>> http://people.freebsd.org/~bapt/Makefile.inc1.diff > >>> Extending the same thing xi-cross-tools and xb-cross-tools (expect the > >>> WITH_GNUCXX=3Dt because it it not set in src.opts.mk when it imho sho= uld.) > >>=20 > >> The patch looks good to my eye. Did you want me to expand it, or do yo= u want to > >> do the honors? > >>=20 > >> About the rest=E2=80=A6 Yea, you may be right=E2=80=A6. MK_GNUCXX is = an odd duck, and that=E2=80=99s > >> likely the problem that should be fixed in a different way. It is real= ly an internal > >> variable that should be set based on the actual compiler type (possibl= y with an > >> override for the odd-duck pair of clang and libstdc++ which may not be= worth > >> supporting). It is telling us we=E2=80=99re doing something horribly w= rong and we should listen > >> to that rather than add another compiler-related kludge to the build s= ystem. I=E2=80=99ll work > >> on that bit. > >>=20 > >> Also an aside: The horribly long command line was to try to get to the= bottom of > >> the breakage, not to promote it as the proper way of doing things. > >=20 > > I haven't committed because it isn't complete yet :) > > I have updated it to the following: > > http://people.freebsd.org/~bapt/Makefile.inc1.diff > > but it is not enough as I end up with: > >=20 > > /tmp/tmips//usr/mips64-freebsd/usr/bin/ld: cannot find -lsupc++ > >=20 > > So it goes further but it is not yet enough >=20 > I actually think xdev and a couple others should move to Makefile, and th= e sub targets stay in Makefile.inc and be invoked with the right TARGET and= TARGET_ARCH so that we get the right defaults. >=20 I 100% agree on that :) regards, Bapt --8GpibOaaTibBMecb Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAlO83KAACgkQ8kTtMUmk6EwIjgCZAU3x1kbCQgphxbxLxbMALlv9 KkIAnjXmGNFOxIrKLPNcLmuA+9ln1FC4 =6PPE -----END PGP SIGNATURE----- --8GpibOaaTibBMecb--