Date: Mon, 15 Jun 2015 11:45:00 -0700 From: Adrian Chadd <adrian@freebsd.org> To: Warner Losh <imp@bsdimp.com> Cc: "freebsd-embedded@freebsd.org" <embedded@freebsd.org>, "freebsd-mips@freebsd.org" <freebsd-mips@freebsd.org> Subject: Re: Starting to cross-compile non-base software for MIPS - what it's like Message-ID: <CAJ-Vmo=8oyyWiRBS9EU%2BKicz_jm90gO46cu_QRu9Az2RHU7zRw@mail.gmail.com> In-Reply-To: <B29CEB72-367C-4295-BD93-2A5C417D0553@bsdimp.com> References: <CAJ-VmonSXYdLP_VpGh%2B3N_WfGwDLqWGcuPvOCf4VkOd0pJ5QWg@mail.gmail.com> <B29CEB72-367C-4295-BD93-2A5C417D0553@bsdimp.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 15 June 2015 at 10:23, Warner Losh <imp@bsdimp.com> wrote:
>
>> On Jun 14, 2015, at 1:13 PM, Adrian Chadd <adrian@freebsd.org> wrote:
>>
>> Hi!
>>
>> I've been hacking at cross compiling things on FreeBSD for a while,
>> and I have made some progress. This email is less of a "how you can do
>> it" and more of a "these are the roadblocks that prevent it from
>> working out of the box."
>>
>> I'm using gcc-4.9.2 from ports - there's a cross compiler for
>> mips/mips64, and there's mips-binutils/mips64-binutils. I'm using
>> those to compile world and kernel. The kernel works; world requires
>> some patching to compile. I'll write a post when I have a complete
>> system booting/running from gcc-4.9.2.
>>
>> My test application is dropbear. Ie, I'm cross-compiling dropbear for
>> mips from freebsd/i386.
>>
>> * There are some issues in libgcc - I have a patch that I've emailed
>> out for review. I'll do some more testing and commit it next week when
>> I know it works.
>> * There are some environment variables that need to be set before I
>> compile dropbear. Here's what it looks like:
>>
>> CPU_ARGS=3D"-march=3Dmips32 -msoft-float -Wa,-msoft-float"
>> INCS=3D"-I/home/adrian/work/freebsd/head-embedded-2/root/mips/usr/includ=
e"
>>
>> export CROSS_COMPILE=3Dmips-portbld-freebsd11.0
>> export CC=3D${CROSS_COMPILE}-gcc
>> export CXX=3D${CROSS_COMPILE}-g++
>> #export LD=3D${CROSS_COMPILE}-ld
>> #export AR=3D${CROSS_COMPILE}-ar
>> #export RANLIB=3D${CROSS_COMPILE}-ranlib
>> #export STRIP=3D${CROSS_COMPILE}-strip
>> export LD=3Dmips-freebsd-l
>> export AR=3Dmips-freebsd-ar
>> export RANLIB=3Dmips-freebsd-ranlib
>> export STRIP=3Dmips-freebsd-strip
>> export SYSROOT=3D/home/adrian/work/freebsd/head-embedded-2/root/mips/
>> export CFLAGS=3D"--sysroot=3D$SYSROOT ${CPU_ARGS} ${INCS} -O"
>> export CXXFLAGS=3D"--sysroot=3D$SYSROOT ${CPU_ARGS} ${INCS} -O"
>> export CPPFLAGS=3D"--sysroot=3D$SYSROOT ${CPU_ARGS} ${INCS} -O"
>> export LDFLAGS=3D--sysroot=3D$SYSROOT
>>
>> # now run:
>> # /configure --host=3Dmips-portbld-freebsd11.0
>> # gmake
>>
>> ... then I hit a problem with how we populate links in /usr/lib and
>> such. TL;DR - the symlinks we use are not relative paths, they're
>> absolute paths - and this makes cross building using sysroot /
>> explicit library paths fail to work right. Sigh. In particular, I
>> needed to manually undo the libgcc symlink for it to compile.
>> compilation complained about other libraries, but the resulting
>> binaries did run.
>
> That=E2=80=99s one of about 6 reasons that what we produce isn=E2=80=99t =
a
> sysroot. We need to create a sysroot target that=E2=80=99s very close
> to the xdev targets with all the compiler bits stripped out. We=E2=80=99r=
e
> close though. I have about 1/2 of the patch needed to remove
> xdev entirely and replace it with sysroot.
Sweet.
> You shouldn=E2=80=99t need the INCS args if you have a proper sysroot.
> And the CPU_ARGS is an indication we=E2=80=99re building our port
> incorrectly. This is the same feedback I gave on the kernel
> make changes you suggested.
It looks like a proper sysroot - I think it's just the dynamic sysroot
stuff being broken.
> I=E2=80=99ve had hacks like this for =E2=80=9Csimple=E2=80=9D arts for ye=
ars. It=E2=80=99s why I wrote
> xdev in the first place. I had it working in the FreeBSD 6 time frame
> for cross building some ports. But once you get past a certain
> class of very well behaved ports, it goes to hell in a hurry.
>> * I'll try to wrangle juli/bsdimp to review the kernel changes needed
>> to compile up the kernel with gcc-4.9.
>
> I=E2=80=99ve already reviewed that one. gcc isn=E2=80=99t being compiled =
correctly
> and that need to be fixed. This is the forth time I=E2=80=99ve given this
> feedback.
There are some other issues too. The softfloat/hardfloat stuff reared
its head - in order to avoid assembler warnings we have to manually
set it, but then the floating point register accesses in fp.S /
swtch.S generate errors. I've worked around that for now, but I need
to dig into why the warnings are generated in the first case.
>> * I'd like some help / comments on what we can do about the absolute
>> symlinks used for various things inside an installed root. Ideally
>> we'd have an option that would let us populate only relative links -
>> that'd instantly make this problem go away.
>
> All installworld-ish targets should use relative path symbolic links.
> The fact they are using absolute paths is a bug. In the FreeBSD 6
> time frame, I fought this battle at work and it wasn=E2=80=99t hard to fi=
x. I
> no longer have access to those sources, and I honestly can=E2=80=99t
> recall how I fixed it (it may have been in a post-processing
> step).
I'd love to get this fixed and into the tree.
>> * I'll go hit up bapt and some netbsd friends for some help with
>> expected sysroot behaviour on gcc/binutils. It's possible there are
>> still bugs with the dynamic sysroot (ie, when you don't compile your
>> toolchain with an explicit sysroot to something other than '/') that
>> need addressing.
>
> bapt and I have been talking about this for a while. We=E2=80=99ve both b=
een
> busy doing other things so haven=E2=80=99t focused on it other than in of=
f
> moments.
>
>> Then I'll worry about starting to convert a few ports to be native
>> cross building.
>
> You should talk to bapt about the rather extensive work he=E2=80=99s done
> in this area, which is why I=E2=80=99ve looked at transitioning the xdev
> support to just sysroot support.
I have been too! fear not. That's how I got this far!
-a
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAJ-Vmo=8oyyWiRBS9EU%2BKicz_jm90gO46cu_QRu9Az2RHU7zRw>