From owner-freebsd-arch@FreeBSD.ORG Thu Feb 28 06:02:51 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 4E909277 for ; Thu, 28 Feb 2013 06:02:51 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-ia0-x236.google.com (mail-ia0-x236.google.com [IPv6:2607:f8b0:4001:c02::236]) by mx1.freebsd.org (Postfix) with ESMTP id 1A59AD22 for ; Thu, 28 Feb 2013 06:02:51 +0000 (UTC) Received: by mail-ia0-f182.google.com with SMTP id k38so1216276iah.41 for ; Wed, 27 Feb 2013 22:02:50 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:sender:subject:mime-version:content-type:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to:x-mailer:x-gm-message-state; bh=xrC209zEoNypFIcAkm7gWS8jByU+3w3KtzV7C10JAmU=; b=PV0E1po6xD0C2sl5RXBvvYCfBzFUOyJyqjX+zqUTe6RufPebgtcenJDYgT9B0wlXS3 KuN5YnywYBToovOWe5ZDkX30GbcAlAWec9y6g71hBOVWn8o0ar1LLvcobQx2o4BaMVQL jZ2MAKxRbkWG0I7zxIH1aljCUdUILkTTrabKRTDuIa8r44R475h2Ezg6RdbnFnucdwrV zkQCCNsM810lxmrwKCsU36CWRBBe7kwirLETiET1XcnX0Kjvhmp4iDJU3a4bYA/PQSwd q+grrD4YAtdgcP6UPpaA4AKpd3xl23g4Kzo/1p8aGpHhtFZ+mCrDR3S+ff/TpRrZj9+V kO6Q== X-Received: by 10.50.155.168 with SMTP id vx8mr9329188igb.73.1362031370077; Wed, 27 Feb 2013 22:02:50 -0800 (PST) Received: from 53.imp.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPS id xe9sm8676369igb.7.2013.02.27.22.02.47 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 27 Feb 2013 22:02:48 -0800 (PST) Sender: Warner Losh Subject: Re: [RFC] external compiler support Mime-Version: 1.0 (Apple Message framework v1085) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: <20130228000241.GF19594@lor.one-eyed-alien.net> Date: Wed, 27 Feb 2013 23:02:46 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20130227003517.GB7348@lor.one-eyed-alien.net> <28404C12-67F3-44F0-AB28-02B749472873@bsdimp.com> <51BB3E17-128A-4989-B272-D8B40D4B854B@bsdimp.com> <20130227190804.GB17489@lor.one-eyed-alien.net> <13FB8CB0-9937-4BD8-AE89-0D24494D8663@bsdimp.com> <20130227214445.GA19594@lor.one-eyed-alien.net> <1CC1DB5A-E87A-456C-AD2C-E203146BB736@bsdimp.com> <20130227221552.GC19594@lor.one-eyed-alien.net> <20130228000241.GF19594@lor.one-eyed-alien.net> To: Brooks Davis X-Mailer: Apple Mail (2.1085) X-Gm-Message-State: ALoCoQk1g0GZKAtTithGyVhzPHX/MRlPgK1rAEDDnjetrYxY/eyfwSpLIGd+1gUq7NlDlmrEXBER Cc: freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Feb 2013 06:02:51 -0000 On Feb 27, 2013, at 5:02 PM, Brooks Davis wrote: > On Wed, Feb 27, 2013 at 04:15:52PM -0600, Brooks Davis wrote: >> On Wed, Feb 27, 2013 at 02:47:59PM -0700, Warner Losh wrote: >>>=20 >>> On Feb 27, 2013, at 2:44 PM, Brooks Davis wrote: >>>>>> As a strawman, let's say we add a CROSS_COMPILER_PATH and a >>>>>> CROSS_BINUTILS_PATH. The former will set XCC, XCXX, and XCPP if = they >>>>>> are unset. The latter will control -B and set the various = binutils >>>>>> variables (XNM, XLD, etc). >>>>>=20 >>>>> I'm not sure I like splitting things like that. It is unnatural. >>>>=20 >>>> That's the traditional view with lots of historic merit. At least = in >>>> the short term it's not a useful view for me. I want to be able to >>>> use our existing infrastructure to build a cross binutils and then = use >>>> it with an external compiler. In a clang world, we currently have = one >>>> compiler and many binutils unless we gratuitously build many = compilers >>>> as the FreeBSD build system currently does. Some day we will = likely have >>>> an all-llvm toolchain available and then we will have one toolchain = for >>>> all supported architectures. >>>>=20 >>>> I suppose could hack what I want to do into the traditional single >>>> toolchain world view by build a mips64 xdev toolchain and then = building >>>> a linkfarm and/or set of wrapper scripts to it and the clang I want = to >>>> include, but that seems problematic from a reproducability = perspective >>>> (not to mention performance if I need wrappers to set -B). >>>>=20 >>>> Having a CROSS_TOOLCHAIN_PATH path that sets both would probably be = a >>>> useful compromise in this regard. >>>=20 >>> Are you suggesting something like: >>>=20 >>> CROSS_COMPILER_PATH?=3D${CROSS_TOOLCHAIN_PATH} >>> CROSS_BINUTILS_PATH?=3D${CROSS_TOOLCHAIN_PATH} >>>=20 >>> If so, I'd agree, that would be a very useful compromise: hits my = ease of use issues, and lets you do what you need on the theory that = others will likely need it too. >>=20 >> That's exactly what I'm thinking. >=20 > Here's a rework with that implemented. I'm not 100% sure I got the = list > of binutils right, but it notionally supports both my usecase and a = more > classic cross compiler set. >=20 > -- Brooks >=20 > MFP4 222356, 222371, 222375, 222391, 222403, 222407, 222411, 222446 >=20 > Add support for an external cross compiler. The cross compiler is > specified by passing the XCC, XCXX, and XCPP variables (corresponding = to > CC, CXX, and CPP) to buildworld/buildkernel. The compiler must be = clang > or be configured to target the appropriate architecture. >=20 > To speed build times, if XCC is an absolute path or > WITHOUT_CROSS_COMPILER is defined then no cross compiler will be built > during the cross-tools stage. >=20 > To facilitate the use of unmodified external compilers, a > WITHOUT_FORMAT_EXTENSIONS option is available to supress printf format > checking. >=20 > As a short-term measure, supress a few new clang warnings during = kernel > builds. >=20 > Sponsored by: DARPA, AFRL > Reviewed by: xxx Looks a lot better, some minor comments.. > --- ../../freebsd/src/Makefile.inc1 2013-02-26 21:31:09.000000000 = +0000 > +++ ./Makefile.inc1 2013-02-27 23:31:46.000000000 +0000 > @@ -280,15 +280,61 @@ > .if ${MK_CDDL} =3D=3D "no" > WMAKEENV+=3D NO_CTF=3D1 > .endif > -.if ${CC:T:Mgcc} =3D=3D "gcc" > + > +.if defined(CROSS_TOOLCHAIN_PATH) > +CROSS_COMPILER_PATH?=3D${CROSS_TOOLCHAIN_PATH} > +CROSS_BINUTILS_PATH?=3D${CROSS_TOOLCHAIN_PATH} > +.endif > +XCOMPILERS=3D CC CXX CPP > +.for COMPILER in ${XCOMPILERS} > +.if defined(CROSS_COMPILER_PATH) > +X${COMPILER}?=3D ${CROSS_COMPILER_PATH}/${${COMPILER}} This makes it harder to support FreeBSD make xdev-built compilers. Can = you lose the / and require it to be specified if someone wants to use = just a path? > +.else > +X${COMPILER}?=3D ${${COMPILER}} > +.endif > +.endfor > +XBINUTILS=3D AS AR LD NM OBJDUMP RANLIB STRINGS > +.for BINUTIL in ${XBINUTILS} > +.if defined(CROSS_BINUTILS_PATH) > +X${BINUTIL}?=3D ${CROSS_BINUTILS_PATH}/${${BINUTIL}} Same. > +.else > +X${BINUTIL}?=3D ${${BINUTIL}} > +.endif > +.endfor > +WMAKEENV+=3D CC=3D"${XCC} ${XFLAGS}" CXX=3D"${XCXX} ${XFLAGS}" \ > + CPP=3D"${XCPP} ${XFLAGS}" \ > + AS=3D"${XAS}" AR=3D"${XAR}" LD=3D"${XLD}" NM=3D${XNM} \ > + OBJDUMP=3D${XOBJDUMP} RANLIB=3D${RANLIB} = STRINGS=3D${XSTRINGS} XRANLIB maybe? > + > +.if ${XCC:T:Mgcc} =3D=3D "gcc" > WMAKE_COMPILER_TYPE=3D gcc > -.elif ${CC:T:Mclang} =3D=3D "clang" > +.elif ${XCC:T:Mclang} =3D=3D "clang" > WMAKE_COMPILER_TYPE=3D clang > .elif ${MK_CLANG_IS_CC} =3D=3D "no" > WMAKE_COMPILER_TYPE=3D gcc > .else > WMAKE_COMPILER_TYPE=3D clang > .endif > + > +.if ${XCC:M/*} > +XFLAGS=3D --sysroot=3D${WORLDTMP} > +.if defined(CROSS_BINUTILS_PATH) > +XFLAGS+=3D -B${CROSS_BINUTILS_PATH} Hmmm, but this would need to change somehow... > +.else > +XFLAGS+=3D -B${WORLDTMP}/usr/bin > +.endif > +.if ${TARGET_ARCH} !=3D ${MACHINE_ARCH} && ${WMAKE_COMPILER_TYPE} =3D=3D= "clang" > +.if (${TARGET_ARCH} =3D=3D "arm" || ${TARGET_ARCH} =3D=3D "armv6") && = \ > +${MK_ARM_EABI} !=3D "no" > +TARGET_ABI=3D gnueabi > +.else > +TARGET_ABI=3D unknown > +.endif > +TARGET_TRIPLE?=3D = ${TARGET_ARCH:C/amd64/x86_64/}-${TARGET_ABI}-freebsd10.0 > +XFLAGS+=3D -target ${TARGET_TRIPLE} > +.endif > +.endif > + > WMAKEENV+=3D COMPILER_TYPE=3D${WMAKE_COMPILER_TYPE} > WMAKE=3D ${WMAKEENV} ${MAKE} ${WORLD_FLAGS} -f = Makefile.inc1 DESTDIR=3D${WORLDTMP} >=20 > @@ -321,6 +367,7 @@ >=20 >=20 > LIB32FLAGS=3D -m32 ${LIB32CPUFLAGS} -DCOMPAT_32BIT \ > + --sysroot=3D${WORLDTMP} \ > -isystem ${LIB32TMP}/usr/include/ \ > -L${LIB32TMP}/usr/lib32 \ > -B${LIB32TMP}/usr/lib32 > @@ -336,8 +383,8 @@ > SHLIBDIR=3D/usr/lib32 \ > COMPILER_TYPE=3D${WMAKE_COMPILER_TYPE} > LIB32WMAKEFLAGS+=3D \ > - CC=3D"${CC} ${LIB32FLAGS}" \ > - CXX=3D"${CXX} ${LIB32FLAGS}" \ > + CC=3D"${XCC} ${LIB32FLAGS}" \ > + CXX=3D"${XCXX} ${LIB32FLAGS}" \ > DESTDIR=3D${LIB32TMP} \ > -DCOMPAT_32BIT \ > -DLIBRARIES_ONLY \ > @@ -1284,10 +1331,13 @@ > .endif > .endif >=20 > -.if ${MK_BINUTILS} !=3D "no" > +.if ${XAS:M/*} =3D=3D "" && ${MK_BINUTILS} !=3D "no" > _binutils=3D gnu/usr.bin/binutils > .endif >=20 > +# If an full path to an external cross compiler is given, don't build > +# a cross compiler. > +.if ${XCC:M/*} =3D=3D "" && ${MK_CROSS_COMPILER} !=3D "no" > .if ${MK_CLANG} !=3D "no" && (${MK_CLANG_IS_CC} !=3D "no" || = ${CC:T:Mclang} =3D=3D "clang") > _clang=3D usr.bin/clang > _clang_libs=3D lib/clang > @@ -1296,6 +1346,7 @@ > .if ${MK_GCC} !=3D "no" && (${MK_CLANG_IS_CC} =3D=3D "no" || ${TARGET} = =3D=3D "pc98") > _cc=3D gnu/usr.bin/cc > .endif > +.endif >=20 > cross-tools: > .for _tool in \ > --- ../../freebsd/src/share/mk/bsd.own.mk 2013-02-15 = 18:49:13.000000000 +0000 > +++ share/mk/bsd.own.mk 2013-02-26 21:10:50.000000000 +0000 > @@ -262,6 +262,7 @@ > CAPSICUM \ > CDDL \ > CPP \ > + CROSS_COMPILER \ > CRYPT \ > CTM \ > CVS \ > @@ -271,6 +272,7 @@ > ED_CRYPTO \ > EXAMPLES \ > FLOPPY \ > + FORMAT_EXTENSIONS \ > FORTH \ > FP_LIBC \ > FREEBSD_UPDATE \ > --- ../../freebsd/src/sys/conf/kern.mk 2012-11-11 = 22:15:16.000000000 +0000 > +++ sys/conf/kern.mk 2013-02-26 20:35:48.000000000 +0000 > @@ -5,7 +5,7 @@ > # > CWARNFLAGS?=3D -Wall -Wredundant-decls -Wnested-externs = -Wstrict-prototypes \ > -Wmissing-prototypes -Wpointer-arith -Winline = -Wcast-qual \ > - -Wundef -Wno-pointer-sign -fformat-extensions \ > + -Wundef -Wno-pointer-sign ${FORMAT_EXTENTIONS} \ > -Wmissing-include-dirs -fdiagnostics-show-option \ > ${CWARNEXTRA} > # > @@ -29,7 +29,18 @@ > # enough to error out the whole kernel build. Display them anyway, so = there is > # some incentive to fix them eventually. > CWARNEXTRA?=3D -Wno-error-tautological-compare = -Wno-error-empty-body \ > - -Wno-error-parentheses-equality > + -Wno-error-parentheses-equality \ > + -Wno-sizeof-pointer-memaccess \ > + -Wno-unused-command-line-argument \ > + ${NO_WFORMAT} > +.endif > + > +# External compilers may not support our format extensions. Allow = them > +# to be disabled. WARNING: format checking is disabled in this case. > +.if ${MK_FORMAT_EXTENSIONS} =3D=3D "no" > +NO_WFORMAT=3D -Wno-format > +.else > +FORMAT_EXTENTIONS=3D -fformat-extensions > .endif >=20 > # > diff -uN ../../freebsd/src/tools/build/options/WITHOUT_CROSS_COMPILER = tools/build/options/WITHOUT_CROSS_COMPILER > --- ../../freebsd/src/tools/build/options/WITHOUT_CROSS_COMPILER = 1970-01-01 00:00:00.000000000 +0000 > +++ tools/build/options/WITHOUT_CROSS_COMPILER 2013-02-26 = 21:10:50.000000000 +0000 > @@ -0,0 +1,3 @@ > +.\" $FreeBSD$ > +Set to not build a cross compiler in the cross-tools stage of > +buildworld, buildkernel, etc. > diff -uN = ../../freebsd/src/tools/build/options/WITHOUT_FORMAT_EXTENSIONS = tools/build/options/WITHOUT_FORMAT_EXTENSIONS > --- ../../freebsd/src/tools/build/options/WITHOUT_FORMAT_EXTENSIONS = 1970-01-01 00:00:00.000000000 +0000 > +++ tools/build/options/WITHOUT_FORMAT_EXTENSIONS 2013-02-26 = 20:35:48.000000000 +0000 > @@ -0,0 +1,5 @@ > +.\" $FreeBSD$ > +Set to not enable > +.Fl fformat-extensions > +when compiling the kernel. > +Also disables all format checking.