Date: Tue, 15 Dec 2015 10:02:30 -0700 From: Ian Lepore <ian@freebsd.org> To: Karl Denninger <karl@denninger.net>, "freebsd-arm@freebsd.org" <freebsd-arm@freebsd.org> Subject: Re: Updating / keeping current strategies? Message-ID: <1450198950.25138.50.camel@freebsd.org> In-Reply-To: <56703E2D.8050504@denninger.net> References: <5666F37C.4060908@denninger.net> <10BBDDBB-75AA-4034-B494-9EB28D009882@gromit.dlib.vt.edu> <566EEAC4.1060306@denninger.net> <566F0238.9060605@denninger.net> <1450190791.25138.30.camel@freebsd.org> <567031C9.2090109@denninger.net> <1450195123.25138.42.camel@freebsd.org> <56703E2D.8050504@denninger.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, 2015-12-15 at 10:22 -0600, Karl Denninger wrote: > On 12/15/2015 09:58, Ian Lepore wrote: > > On Tue, 2015-12-15 at 09:29 -0600, Karl Denninger wrote: > > > On 12/15/2015 08:46, Ian Lepore wrote: > > > > > > There seem to be some problems with this in the general > > > > > > sense, > > > > > > unless > > > > > > you're willing to move the media around (I'm generally > > > > > > not.) > > > > > > > > > > > > Using that script in the Wiki appears to work, but there > > > > > > are > > > > > > problems. > > > > > > If you try to nfs mount the object and source directories > > > > > > on > > > > > > the > > > > > > target > > > > > > so you can do a "make installworld" or "make installkernel" > > > > > > on > > > > > > the > > > > > > target you quickly find that clang wasn't cross-built in > > > > > > the > > > > > > object > > > > > > directory; the "tmp" directory in object contains a copy of > > > > > > it, > > > > > > but > > > > > > it's > > > > > > compiled for your _*source*_ machine (in this case > > > > > > FreeBSD/amd64) > > > > > > and > > > > > > the install blows up as it can't determine the cc version. > > > > > > The > > > > > > interesting thing is that what's in ~/nfsroot/usr/bin/cc is > > > > > > correct > > > > > > which doesn't appear to make sense, but it is what it is. > > > > > > > > > > > > I'm going to see if I can get a rsync-type thing to > > > > > > work.... > > > > > > > > > > > It.... gets.... worse. > > > > > > > > > > I can update using rsync. However, the armv6hf target given > > > > > in > > > > > the > > > > > wiki > > > > > at https://wiki.freebsd.org/FreeBSD/arm/crossbuild produces > > > > > binaries > > > > > that blow up when floating point is attempted. Armv6 is > > > > > deprecated > > > > > and > > > > > won't build at all. > > > > > > > > > I'm having a hard time understanding this. Nobody else is > > > > reporting > > > > problems with cross-built binaries that use floating point. > > > > Also, > > > > armv6 is deprecated only in the sense that armv6hf is more > > > > efficient > > > > and there's no reason to use the old softfp ABI anymore. But > > > > that > > > > doesn't mean it can't be built, it builds and cross-builds just > > > > fine. > > > > > > > > -- Ian > > > Well, I'm reporting what I got. I followed the instructions in > > > the > > > Wiki, got a product nfsroot directory and the expected src and > > > obj > > > directories, attempted to mount the src and obj directories over > > > nfs > > > and > > > execute a "make installworld" on the target and that blew up with > > > complaints about expected directories being missing. > > > > > Oh, you can't do that. The pathnames in the obj directory are > > correct > > for the build machine, but wrong when nfs-mounted on another > > machine. > Well, but they are if I maintain the paths. > > Example: I have the build at /pics/CrossBuild on the source (10.2 > -STABLE > amd64) machine. > I export /pics/CrossBuild. > > I then mount that at /pics/CrossBuild on the target. Now the paths > are > the same. > > But..... when I try to do a "make installkernel" as you would do if > doing this on a consistent architecture what I get is this: > > root@Pool-MCP:/pics/CrossBuild # ./mk installkernel > TARGET_ARCH=armv6hf > + cd /pics/CrossBuild/src > + nice -10 make -j 4 -DNO_CLEAN 'TARGET_ARCH=armv6hf' > 'DESTDIR=/pics/CrossBuild/nfsroot' > '__MAKE_CONF=/pics/CrossBuild/config/make.conf' > 'srcconf=/pics/CrossBuild/config/src.conf' 'KERNCONF=RPI2' > 'UBLDR_LOADADDR=0x2000000' installkernel 'TARGET_ARCH=armv6hf' > --- installkernel --- > --- installkernel --- > -------------------------------------------------------------- > > > > Installing kernel RPI2 > -------------------------------------------------------------- > cd /pics/CrossBuild/obj/arm.armv6hf/pics/CrossBuild/src/sys/RPI2; > MAKEOBJDIRPREFIX=/pics/CrossBuild/obj/arm.armv6hf > MACHINE_ARCH=armv6hf > MACHINE=arm CPUTYPE= > GROFF_BIN_PATH=/pics/CrossBuild/obj/arm.armv6hf/pics/CrossBuild/src/t > mp/legacy/usr/bin > GROFF_FONT_PATH=/pics/CrossBuild/obj/arm.armv6hf/pics/CrossBuild/src/ > tmp/legacy/usr/share/groff_font > GROFF_TMAC_PATH=/pics/CrossBuild/obj/arm.armv6hf/pics/CrossBuild/src/ > tmp/legacy/usr/share/tmac > CC="cc " CXX="c++ " DEPFLAGS="" CPP="cpp " AS="as" AR="ar" > LD="ld" > NM=nm OBJDUMP=objdump OBJCOPY="objcopy" RANLIB=ranlib STRINGS= > SIZE="size" > PATH=/pics/CrossBuild/obj/arm.armv6hf/pics/CrossBuild/src/tmp/legacy/ > usr/sbin:/pics/CrossBuild/obj/arm.armv6hf/pics/CrossBuild/src/tmp/leg > acy/usr/bin:/pics/CrossBuild/obj/arm.armv6hf/pics/CrossBuild/src/tmp/ > legacy/bin:/pics/CrossBuild/obj/arm.armv6hf/pics/CrossBuild/src/tmp/u > sr/sbin:/pics/CrossBuild/obj/arm.armv6hf/pics/CrossBuild/src/tmp/usr/ > bin:/sbin:/bin:/usr/sbin:/usr/bin > make KERNEL=kernel install > sh: cc: Exec format error > make[2]: "/pics/CrossBuild/src/share/mk/bsd.compiler.mk" line 137: > Unable to determine compiler type for cc . Consider setting > COMPILER_TYPE. > *** [installkernel] Error code 1 > > make[1]: stopped in /pics/CrossBuild/src > 1 error > > make[1]: stopped in /pics/CrossBuild/src > *** [installkernel] Error code 2 > > make: stopped in /pics/CrossBuild/src > 1 error > > make: stopped in /pics/CrossBuild/src > root@Pool-MCP:/pics/CrossBuild # > > When investigating this what I find is that the executable in the > path > for "cc" is in fact an arm64 executable! So I have a built system > but > no way to install it. > > > > > > The instructions in the wiki page you cited are how to set up a > > development environment that uses an nfs-mounted root directory. > > You > > can also use it as a source for rsync, but you can't install > > directly > > from the objdir of such a build environment using an nfs mount > > unless > > the target system has the exact directory layout of the build > > system > > after nfs-mounting the source and obj dirs. > Except that doesn't work because the PATH contains executables for > the > source which is a different architecture. > > And when I tried to rsync it I got a system that booted and ran, sort > of, but anything that did floating point blew up. That might be > because > of schg flags prevented some of the copies and rsync has no good way > to > get around that (removing them from the target is undesirable for all > the obvious reasons.) > > > > > Another way to make it work is to nfs-mount the target's root > > directory > > on your build machine, then you can make installworld > > DESTDIR=<mountpt> > That's an interesting idea that I will look into. It's undesirable > for a > number of reasons down the road (enabling nfs server on these > machines > isn't my idea of a good thing over time) but it might get me going > for > the moment. > > > I then attempted to rsync the nfsroot directory to the target > > > machine > > > (a > > > Raspberry Pi2); that succeeded. But, once the machine was > > > rebooted > > > any > > > attempt to execute anything that had floating point operations > > > dumped; > > > "awk", for example, blows up. > > > > > I built on a 10.2-STABLE machine, if that matters. > > Using a kernel and world built yesterday for armv6hf... > > > > root@wand :~ # awk "BEGIN {print 2.7+4.2 }" > > 6.9 > > > > I do all my building on a 10-stable machine. > > > > I wonder if your first attempt to installworld somehow installed > > some > > x86 stuff by accident and now you've got a mixed/broken world, and > > rsyn > > c didn't fix it because some of the broken files were more recent? > > > > -- Ian > Maybe. But I don't think so; I suspect the issue revolves around > schg > flags (in other words some things didn't update) and if so it's a > problem since I don't believe rsync can deal with that. > I had forgotten about the "can't determine compiler" error, other people have reported that too. It's another roadblock that adds up to "you can't do that" (install a crossbuilt system by mounting the objdir on the target). The way the crossbuild works is to create a bunch of cross-tools and put them in a PATH that makes them get used before the standard system tools; they're still there at install time. I'm not sure it's impossible to cross-build then non-cross install, but it certainly can't be done with the way the build system currently works. At $work we deal with updating by having readonly rootfs and all modified data on a different filesystem. We have two slices for rootfs so if we're running from slice 1 we newfs then populate slice 2, switch the active flag in the MBR, and reboot. Warner has recently been adding similar features to nanobsd, so it may be available as a solution for this stuff pretty soon. -- Ian
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1450198950.25138.50.camel>