From owner-freebsd-mips@FreeBSD.ORG Mon Jun 15 17:23:05 2015 Return-Path: Delivered-To: freebsd-mips@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 240A4D4A for ; Mon, 15 Jun 2015 17:23:05 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-ig0-f182.google.com (mail-ig0-f182.google.com [209.85.213.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E1FC7B3F for ; Mon, 15 Jun 2015 17:23:04 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: by igblz2 with SMTP id lz2so55542654igb.1 for ; Mon, 15 Jun 2015 10:23:04 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:mime-version:content-type:from :in-reply-to:date:cc:message-id:references:to; bh=Jg61YZFHp+UqzSewoJRQcqrihKrefHlpbVbiDNM4r5g=; b=lS6zYDer04pUAdumKJwaZVjcEH3UmsrM8fmqTxZpuT8aF9nnz2lnjA38I9YL5Vx0jj WHPtbo6MLuD8MArfTaa7SvAHmZSjC/IoCfmmjERVPcZIbYZx7WAtKQ0FhojzhRLOpcDT OhMuBujpbZ4edvruTolll/euLkzo/FcgWBSNhrHktvDYiWuvCpb+FtM+K7LSC45lfEQ7 2WDc5n8lRBhMB1SoO7BrUst5QyVzpt2HSrE/TCYly+uhkX2hDSeuUs1ckrVZ9nOVY0BM p4Q2Y9TBJ272XzRzKzhRGil9BkBOPmXxtU6C6OYy2wvugt69HymS9sS3z8jU8mEQXRLL o7Qg== X-Gm-Message-State: ALoCoQkzH3u1ul55VAXtBriyWMBD8PKbQ8suiiWoPOmVHknhYtI0oqevw/U75IwrGmviKHquMJxX X-Received: by 10.50.79.196 with SMTP id l4mr22545722igx.48.1434388984250; Mon, 15 Jun 2015 10:23:04 -0700 (PDT) Received: from netflix-mac-wired.bsdimp.com ([50.253.99.174]) by mx.google.com with ESMTPSA id rt5sm7993918igb.8.2015.06.15.10.23.03 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 15 Jun 2015 10:23:03 -0700 (PDT) Sender: Warner Losh Subject: Re: Starting to cross-compile non-base software for MIPS - what it's like Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2098\)) Content-Type: multipart/signed; boundary="Apple-Mail=_2B777C26-1D06-4FE9-96C4-978352D20094"; protocol="application/pgp-signature"; micalg=pgp-sha512 X-Pgp-Agent: GPGMail 2.5b6 From: Warner Losh In-Reply-To: Date: Mon, 15 Jun 2015 11:23:00 -0600 Cc: "freebsd-embedded@freebsd.org" , "freebsd-mips@freebsd.org" Message-Id: References: To: Adrian Chadd X-Mailer: Apple Mail (2.2098) X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Jun 2015 17:23:05 -0000 --Apple-Mail=_2B777C26-1D06-4FE9-96C4-978352D20094 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Jun 14, 2015, at 1:13 PM, Adrian Chadd wrote: >=20 > Hi! >=20 > I've been hacking at cross compiling things on FreeBSD for a while, > and I have made some progress. This email is less of a "how you can do > it" and more of a "these are the roadblocks that prevent it from > working out of the box." >=20 > I'm using gcc-4.9.2 from ports - there's a cross compiler for > mips/mips64, and there's mips-binutils/mips64-binutils. I'm using > those to compile world and kernel. The kernel works; world requires > some patching to compile. I'll write a post when I have a complete > system booting/running from gcc-4.9.2. >=20 > My test application is dropbear. Ie, I'm cross-compiling dropbear for > mips from freebsd/i386. >=20 > * There are some issues in libgcc - I have a patch that I've emailed > out for review. I'll do some more testing and commit it next week when > I know it works. > * There are some environment variables that need to be set before I > compile dropbear. Here's what it looks like: >=20 > CPU_ARGS=3D"-march=3Dmips32 -msoft-float -Wa,-msoft-float" > = INCS=3D"-I/home/adrian/work/freebsd/head-embedded-2/root/mips/usr/include"= >=20 > export CROSS_COMPILE=3Dmips-portbld-freebsd11.0 > export CC=3D${CROSS_COMPILE}-gcc > export CXX=3D${CROSS_COMPILE}-g++ > #export LD=3D${CROSS_COMPILE}-ld > #export AR=3D${CROSS_COMPILE}-ar > #export RANLIB=3D${CROSS_COMPILE}-ranlib > #export STRIP=3D${CROSS_COMPILE}-strip > export LD=3Dmips-freebsd-l > export AR=3Dmips-freebsd-ar > export RANLIB=3Dmips-freebsd-ranlib > export STRIP=3Dmips-freebsd-strip > export SYSROOT=3D/home/adrian/work/freebsd/head-embedded-2/root/mips/ > export CFLAGS=3D"--sysroot=3D$SYSROOT ${CPU_ARGS} ${INCS} -O" > export CXXFLAGS=3D"--sysroot=3D$SYSROOT ${CPU_ARGS} ${INCS} -O" > export CPPFLAGS=3D"--sysroot=3D$SYSROOT ${CPU_ARGS} ${INCS} -O" > export LDFLAGS=3D--sysroot=3D$SYSROOT >=20 > # now run: > # /configure --host=3Dmips-portbld-freebsd11.0 > # gmake >=20 > ... then I hit a problem with how we populate links in /usr/lib and > such. TL;DR - the symlinks we use are not relative paths, they're > absolute paths - and this makes cross building using sysroot / > explicit library paths fail to work right. Sigh. In particular, I > needed to manually undo the libgcc symlink for it to compile. > compilation complained about other libraries, but the resulting > binaries did run. That=E2=80=99s one of about 6 reasons that what we produce isn=E2=80=99t = a sysroot. We need to create a sysroot target that=E2=80=99s very close to the xdev targets with all the compiler bits stripped out. We=E2=80=99re= close though. I have about 1/2 of the patch needed to remove xdev entirely and replace it with sysroot. You shouldn=E2=80=99t need the INCS args if you have a proper sysroot. And the CPU_ARGS is an indication we=E2=80=99re building our port incorrectly. This is the same feedback I gave on the kernel make changes you suggested. > * yes, I tested the dropbear/dbclient binaries in a mips qemu VM (not > qemu-mips-user, but a full mips MALTA VM.) Both worked fine. >=20 > * the sysroot setup didn't correctly set the include path stuff right > - I need to chase that down. Ie, it was still searching in > /usr/include/ rather than ${SYSROOT}/usr/include/ . The library code > obeyed the library searching, except for that hardlink. That sounds like a bug in the default path of our gcc port. > So, the very basics of "how can we cross compile things to run inside > a freebsd-mips machine" work fine. It's amusing and annoying that the > sysroot thing popped up. I do wonder if this class of problem keeps > popping up when people do builds from one freebsd version to another; > but I currently have no clear idea how to automate detecting this. For one version to another we have a very controlled and constrained environment. Inside of that, there are enough people building in = purposely broken environments to detect regressions. Once you involve configure and foreign software things become rather unconstrained... > On the plus side, it did work. I=E2=80=99ve had hacks like this for =E2=80=9Csimple=E2=80=9D arts for = years. It=E2=80=99s why I wrote xdev in the first place. I had it working in the FreeBSD 6 time frame for cross building some ports. But once you get past a certain class of very well behaved ports, it goes to hell in a hurry. > So: >=20 > * I'll try to wrangle juli/bsdimp to review the libgcc change I have > so userland compiles using gcc-4.9 with warnings disabled. I=E2=80=99ll be happy to take a look. > * I'll try to wrangle juli/bsdimp to review the kernel changes needed > to compile up the kernel with gcc-4.9. I=E2=80=99ve already reviewed that one. gcc isn=E2=80=99t being compiled = correctly and that need to be fixed. This is the forth time I=E2=80=99ve given = this feedback. > * I'd like some help / comments on what we can do about the absolute > symlinks used for various things inside an installed root. Ideally > we'd have an option that would let us populate only relative links - > that'd instantly make this problem go away. All installworld-ish targets should use relative path symbolic links. The fact they are using absolute paths is a bug. In the FreeBSD 6 time frame, I fought this battle at work and it wasn=E2=80=99t hard to = fix. I no longer have access to those sources, and I honestly can=E2=80=99t recall how I fixed it (it may have been in a post-processing step). > * I'll go hit up bapt and some netbsd friends for some help with > expected sysroot behaviour on gcc/binutils. It's possible there are > still bugs with the dynamic sysroot (ie, when you don't compile your > toolchain with an explicit sysroot to something other than '/') that > need addressing. bapt and I have been talking about this for a while. We=E2=80=99ve both = been busy doing other things so haven=E2=80=99t focused on it other than in = off moments. > Then I'll worry about starting to convert a few ports to be native > cross building. You should talk to bapt about the rather extensive work he=E2=80=99s = done in this area, which is why I=E2=80=99ve looked at transitioning the xdev support to just sysroot support. Warner --Apple-Mail=_2B777C26-1D06-4FE9-96C4-978352D20094 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJVfwn1AAoJEGwc0Sh9sBEAMI0P/j12ZOpHoOHhBkn0crUdLTR8 FRskASw84nMM5/h0vOkVlaBVrWxj/pRSbl7ysUhMXWC0CHx3RK5tKqrDby0A7TFj UurUZE1kmD7oUacX/tAMlnZuL5di2jopmqntprY83HvZoUsXsoeoDhTPxOXjW5KB c7UhbCZPCZkxXMNWUFIAPs/DI5HfpfxQxZmtq3HyLPuQOlzQt+Q24nocQzRXxT5b PcKSd0pN+F/Km6CiPcZzdg0oGefuwLZ+oOMrJH/igiqR94WmYypOvksb/E+H5mEa 6kpGDJuzrBueVy/Rx0TRK+MhlsGcM3zFtYmpmU9H06smR9RaYizka2NLqA38gop5 gvoZvELK0050+cjUBFuoS2hM8e258mkphfr+HKAsittZtT1HX7A8OA0W3AID6Vi6 xYvGisMllvHqN9cnGj3pNnbXRqeYx6RAJylmJnnAvGjbTw4W8C6XcD/07kzBxO3M H5qgTnn/STF0TwwlA4o6JxX0XF1mKXtGP8pvBMHun77yfQ1tBTuLEUHrCuK7Uar4 UbPpu0BgOapEhDsVZUK+MiMcUy88n1kxRoBkgzBQXsQ9JMdd2pnOAyQh+xL5tHQg GFg2UeyPRok1EkaJJLGTEXlwp6CT9xq0FufC+b3JBM46qZQLvio9LgxM97nLUDCZ YSSgptmDj+RtNJeJdXHS =J58A -----END PGP SIGNATURE----- --Apple-Mail=_2B777C26-1D06-4FE9-96C4-978352D20094--