From owner-freebsd-arch@FreeBSD.ORG Thu Feb 28 17:23:23 2013 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 97E9A78B for ; Thu, 28 Feb 2013 17:23:23 +0000 (UTC) (envelope-from brooks@lor.one-eyed-alien.net) Received: from lor.one-eyed-alien.net (lor.one-eyed-alien.net [69.66.77.232]) by mx1.freebsd.org (Postfix) with ESMTP id 37E49384 for ; Thu, 28 Feb 2013 17:23:19 +0000 (UTC) Received: from lor.one-eyed-alien.net (localhost [127.0.0.1]) by lor.one-eyed-alien.net (8.14.5/8.14.5) with ESMTP id r1SHNJgo033133; Thu, 28 Feb 2013 11:23:19 -0600 (CST) (envelope-from brooks@lor.one-eyed-alien.net) Received: (from brooks@localhost) by lor.one-eyed-alien.net (8.14.5/8.14.5/Submit) id r1SHNIeX033132; Thu, 28 Feb 2013 11:23:18 -0600 (CST) (envelope-from brooks) Date: Thu, 28 Feb 2013 11:23:18 -0600 From: Brooks Davis To: Warner Losh Subject: Re: [RFC] external compiler support Message-ID: <20130228172318.GB20864@lor.one-eyed-alien.net> References: <20130227003517.GB7348@lor.one-eyed-alien.net> <28404C12-67F3-44F0-AB28-02B749472873@bsdimp.com> <51BB3E17-128A-4989-B272-D8B40D4B854B@bsdimp.com> <20130227190804.GB17489@lor.one-eyed-alien.net> <13FB8CB0-9937-4BD8-AE89-0D24494D8663@bsdimp.com> <20130227214445.GA19594@lor.one-eyed-alien.net> <1CC1DB5A-E87A-456C-AD2C-E203146BB736@bsdimp.com> <20130227221552.GC19594@lor.one-eyed-alien.net> <20130228000241.GF19594@lor.one-eyed-alien.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="QTprm0S8XgL7H0Dt" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Feb 2013 17:23:23 -0000 --QTprm0S8XgL7H0Dt Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Feb 27, 2013 at 11:02:46PM -0700, Warner Losh wrote: >=20 > On Feb 27, 2013, at 5:02 PM, Brooks Davis wrote: >=20 > > On Wed, Feb 27, 2013 at 04:15:52PM -0600, Brooks Davis wrote: > >> On Wed, Feb 27, 2013 at 02:47:59PM -0700, Warner Losh wrote: > >>>=20 > >>> On Feb 27, 2013, at 2:44 PM, Brooks Davis wrote: > >>>>>> As a strawman, let's say we add a CROSS_COMPILER_PATH and a > >>>>>> CROSS_BINUTILS_PATH. The former will set XCC, XCXX, and XCPP if t= hey > >>>>>> are unset. The latter will control -B and set the various binutils > >>>>>> variables (XNM, XLD, etc). > >>>>>=20 > >>>>> I'm not sure I like splitting things like that. It is unnatural. > >>>>=20 > >>>> That's the traditional view with lots of historic merit. At least in > >>>> the short term it's not a useful view for me. I want to be able to > >>>> use our existing infrastructure to build a cross binutils and then u= se > >>>> it with an external compiler. In a clang world, we currently have o= ne > >>>> compiler and many binutils unless we gratuitously build many compile= rs > >>>> as the FreeBSD build system currently does. Some day we will likely= have > >>>> an all-llvm toolchain available and then we will have one toolchain = for > >>>> all supported architectures. > >>>>=20 > >>>> I suppose could hack what I want to do into the traditional single > >>>> toolchain world view by build a mips64 xdev toolchain and then build= ing > >>>> a linkfarm and/or set of wrapper scripts to it and the clang I want = to > >>>> include, but that seems problematic from a reproducability perspecti= ve > >>>> (not to mention performance if I need wrappers to set -B). > >>>>=20 > >>>> Having a CROSS_TOOLCHAIN_PATH path that sets both would probably be a > >>>> useful compromise in this regard. > >>>=20 > >>> Are you suggesting something like: > >>>=20 > >>> CROSS_COMPILER_PATH?=3D${CROSS_TOOLCHAIN_PATH} > >>> CROSS_BINUTILS_PATH?=3D${CROSS_TOOLCHAIN_PATH} > >>>=20 > >>> If so, I'd agree, that would be a very useful compromise: hits my eas= e of use issues, and lets you do what you need on the theory that others wi= ll likely need it too. > >>=20 > >> That's exactly what I'm thinking. > >=20 > > Here's a rework with that implemented. I'm not 100% sure I got the list > > of binutils right, but it notionally supports both my usecase and a more > > classic cross compiler set. > >=20 > > -- Brooks > >=20 > > MFP4 222356, 222371, 222375, 222391, 222403, 222407, 222411, 222446 > >=20 > > Add support for an external cross compiler. The cross compiler is > > specified by passing the XCC, XCXX, and XCPP variables (corresponding to > > CC, CXX, and CPP) to buildworld/buildkernel. The compiler must be clang > > or be configured to target the appropriate architecture. > >=20 > > To speed build times, if XCC is an absolute path or > > WITHOUT_CROSS_COMPILER is defined then no cross compiler will be built > > during the cross-tools stage. > >=20 > > To facilitate the use of unmodified external compilers, a > > WITHOUT_FORMAT_EXTENSIONS option is available to supress printf format > > checking. > >=20 > > As a short-term measure, supress a few new clang warnings during kernel > > builds. > >=20 > > Sponsored by: DARPA, AFRL > > Reviewed by: xxx >=20 > Looks a lot better, some minor comments.. Here's another version that addresses those comments. I decided to rename CROSS_*_PATH to CROSS_*_PREFIX because it offended my sensibilities to call something that may not refer to a filesystem object a path. -- Brooks MFP4 222356, 222371, 222375, 222391, 222403, 222407, 222411, 222446, 222461 Add support for an external cross compiler. The cross compiler is specified by passing the XCC, XCXX, and XCPP variables (corresponding to CC, CXX, and CPP) to buildworld/buildkernel. The compiler must be clang or be configured to target the appropriate architecture. To speed build times, if XCC is an absolute path or WITHOUT_CROSS_COMPILER is defined then no cross compiler will be built during the cross-tools stage. To facilitate the use of unmodified external compilers, a WITHOUT_FORMAT_EXTENSIONS option is available to supress printf format checking. As a short-term measure, supress a few new clang warnings during kernel builds. Sponsored by: DARPA, AFRL Reviewed by: xxx --- ../../freebsd/src/Makefile.inc1 2013-02-26 21:31:09.000000000 +0000 +++ ./Makefile.inc1 2013-02-28 16:44:30.000000000 +0000 @@ -280,15 +280,66 @@ .if ${MK_CDDL} =3D=3D "no" WMAKEENV+=3D NO_CTF=3D1 .endif -.if ${CC:T:Mgcc} =3D=3D "gcc" + +.if defined(CROSS_TOOLCHAIN_PREFIX) +CROSS_COMPILER_PREFIX?=3D${CROSS_TOOLCHAIN_PREFIX} +CROSS_BINUTILS_PREFIX?=3D${CROSS_TOOLCHAIN_PREFIX} +.endif +XCOMPILERS=3D CC CXX CPP +.for COMPILER in ${XCOMPILERS} +.if defined(CROSS_COMPILER_PREFIX) +X${COMPILER}?=3D ${CROSS_COMPILER_PREFIX}${${COMPILER}} +.else +X${COMPILER}?=3D ${${COMPILER}} +.endif +.endfor +XBINUTILS=3D AS AR LD NM OBJDUMP RANLIB STRINGS +.for BINUTIL in ${XBINUTILS} +.if defined(CROSS_BINUTILS_PREFIX) +X${BINUTIL}?=3D ${CROSS_BINUTILS_PREFIX}${${BINUTIL}} +.else +X${BINUTIL}?=3D ${${BINUTIL}} +.endif +.endfor +WMAKEENV+=3D CC=3D"${XCC} ${XFLAGS}" CXX=3D"${XCXX} ${XFLAGS}" \ + CPP=3D"${XCPP} ${XFLAGS}" \ + AS=3D"${XAS}" AR=3D"${XAR}" LD=3D"${XLD}" NM=3D${XNM} \ + OBJDUMP=3D${XOBJDUMP} RANLIB=3D${XRANLIB} STRINGS=3D${XSTRINGS} + +.if ${XCC:T:Mgcc} =3D=3D "gcc" WMAKE_COMPILER_TYPE=3D gcc -.elif ${CC:T:Mclang} =3D=3D "clang" +.elif ${XCC:T:Mclang} =3D=3D "clang" WMAKE_COMPILER_TYPE=3D clang .elif ${MK_CLANG_IS_CC} =3D=3D "no" WMAKE_COMPILER_TYPE=3D gcc .else WMAKE_COMPILER_TYPE=3D clang .endif + +.if ${XCC:M/*} +XFLAGS=3D --sysroot=3D${WORLDTMP} +.if defined(CROSS_BINUTILS_PREFIX) +# In the case of xdev-build tools, CROSS_BINUTILS_PREFIX won't be a +# directory, but the compiler will look in the right place for it's +# tools so we don't need to tell it where to look. +.if exists(${CROSS_BINUTILS_PREFIX}) +XFLAGS+=3D -B${CROSS_BINUTILS_PREFIX} +.endif +.else +XFLAGS+=3D -B${WORLDTMP}/usr/bin +.endif +.if ${TARGET_ARCH} !=3D ${MACHINE_ARCH} && ${WMAKE_COMPILER_TYPE} =3D=3D "= clang" +.if (${TARGET_ARCH} =3D=3D "arm" || ${TARGET_ARCH} =3D=3D "armv6") && \ +${MK_ARM_EABI} !=3D "no" +TARGET_ABI=3D gnueabi +.else +TARGET_ABI=3D unknown +.endif +TARGET_TRIPLE?=3D ${TARGET_ARCH:C/amd64/x86_64/}-${TARGET_ABI}-freebsd10.0 +XFLAGS+=3D -target ${TARGET_TRIPLE} +.endif +.endif + WMAKEENV+=3D COMPILER_TYPE=3D${WMAKE_COMPILER_TYPE} WMAKE=3D ${WMAKEENV} ${MAKE} ${WORLD_FLAGS} -f Makefile.inc1 DESTDIR=3D${= WORLDTMP} =20 @@ -321,6 +372,7 @@ =20 =20 LIB32FLAGS=3D -m32 ${LIB32CPUFLAGS} -DCOMPAT_32BIT \ + --sysroot=3D${WORLDTMP} \ -isystem ${LIB32TMP}/usr/include/ \ -L${LIB32TMP}/usr/lib32 \ -B${LIB32TMP}/usr/lib32 @@ -336,8 +388,8 @@ SHLIBDIR=3D/usr/lib32 \ COMPILER_TYPE=3D${WMAKE_COMPILER_TYPE} LIB32WMAKEFLAGS+=3D \ - CC=3D"${CC} ${LIB32FLAGS}" \ - CXX=3D"${CXX} ${LIB32FLAGS}" \ + CC=3D"${XCC} ${LIB32FLAGS}" \ + CXX=3D"${XCXX} ${LIB32FLAGS}" \ DESTDIR=3D${LIB32TMP} \ -DCOMPAT_32BIT \ -DLIBRARIES_ONLY \ @@ -1284,10 +1336,13 @@ .endif .endif =20 -.if ${MK_BINUTILS} !=3D "no" +.if ${XAS:M/*} =3D=3D "" && ${MK_BINUTILS} !=3D "no" _binutils=3D gnu/usr.bin/binutils .endif =20 +# If an full path to an external cross compiler is given, don't build +# a cross compiler. +.if ${XCC:M/*} =3D=3D "" && ${MK_CROSS_COMPILER} !=3D "no" .if ${MK_CLANG} !=3D "no" && (${MK_CLANG_IS_CC} !=3D "no" || ${CC:T:Mclang= } =3D=3D "clang") _clang=3D usr.bin/clang _clang_libs=3D lib/clang @@ -1296,6 +1351,7 @@ .if ${MK_GCC} !=3D "no" && (${MK_CLANG_IS_CC} =3D=3D "no" || ${TARGET} =3D= =3D "pc98") _cc=3D gnu/usr.bin/cc .endif +.endif =20 cross-tools: .for _tool in \ --- ../../freebsd/src/share/mk/bsd.own.mk 2013-02-15 18:49:13.000000000 +00= 00 +++ share/mk/bsd.own.mk 2013-02-26 21:10:50.000000000 +0000 @@ -262,6 +262,7 @@ CAPSICUM \ CDDL \ CPP \ + CROSS_COMPILER \ CRYPT \ CTM \ CVS \ @@ -271,6 +272,7 @@ ED_CRYPTO \ EXAMPLES \ FLOPPY \ + FORMAT_EXTENSIONS \ FORTH \ FP_LIBC \ FREEBSD_UPDATE \ --- ../../freebsd/src/sys/conf/kern.mk 2012-11-11 22:15:16.000000000 +0000 +++ sys/conf/kern.mk 2013-02-26 20:35:48.000000000 +0000 @@ -5,7 +5,7 @@ # CWARNFLAGS?=3D -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototype= s \ -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual \ - -Wundef -Wno-pointer-sign -fformat-extensions \ + -Wundef -Wno-pointer-sign ${FORMAT_EXTENTIONS} \ -Wmissing-include-dirs -fdiagnostics-show-option \ ${CWARNEXTRA} # @@ -29,7 +29,18 @@ # enough to error out the whole kernel build. Display them anyway, so the= re is # some incentive to fix them eventually. CWARNEXTRA?=3D -Wno-error-tautological-compare -Wno-error-empty-body \ - -Wno-error-parentheses-equality + -Wno-error-parentheses-equality \ + -Wno-sizeof-pointer-memaccess \ + -Wno-unused-command-line-argument \ + ${NO_WFORMAT} +.endif + +# External compilers may not support our format extensions. Allow them +# to be disabled. WARNING: format checking is disabled in this case. +.if ${MK_FORMAT_EXTENSIONS} =3D=3D "no" +NO_WFORMAT=3D -Wno-format +.else +FORMAT_EXTENTIONS=3D -fformat-extensions .endif =20 # diff -uN ../../freebsd/src/tools/build/options/WITHOUT_CROSS_COMPILER tools= /build/options/WITHOUT_CROSS_COMPILER --- ../../freebsd/src/tools/build/options/WITHOUT_CROSS_COMPILER 1970-01-01= 00:00:00.000000000 +0000 +++ tools/build/options/WITHOUT_CROSS_COMPILER 2013-02-26 21:10:50.00000000= 0 +0000 @@ -0,0 +1,3 @@ +.\" $FreeBSD$ +Set to not build a cross compiler in the cross-tools stage of +buildworld, buildkernel, etc. diff -uN ../../freebsd/src/tools/build/options/WITHOUT_FORMAT_EXTENSIONS to= ols/build/options/WITHOUT_FORMAT_EXTENSIONS --- ../../freebsd/src/tools/build/options/WITHOUT_FORMAT_EXTENSIONS 1970-01= -01 00:00:00.000000000 +0000 +++ tools/build/options/WITHOUT_FORMAT_EXTENSIONS 2013-02-26 20:35:48.00000= 0000 +0000 @@ -0,0 +1,5 @@ +.\" $FreeBSD$ +Set to not enable +.Fl fformat-extensions +when compiling the kernel. +Also disables all format checking. --QTprm0S8XgL7H0Dt Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iD8DBQFRL5KGXY6L6fI4GtQRAle4AKCbHWlkXozVQxHwiX8sc/SD8YG73QCfWYHd fnDDmsstIOciqvREFr38OtQ= =MDSJ -----END PGP SIGNATURE----- --QTprm0S8XgL7H0Dt--