From owner-freebsd-stable@FreeBSD.ORG Sat Aug 30 07:20:58 2014 Return-Path: Delivered-To: stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B25366F1; Sat, 30 Aug 2014 07:20:58 +0000 (UTC) Received: from gw.catspoiler.org (cl-1657.chi-02.us.sixxs.net [IPv6:2001:4978:f:678::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 57FFB17CA; Sat, 30 Aug 2014 07:20:58 +0000 (UTC) Received: from FreeBSD.org (mousie.catspoiler.org [192.168.101.2]) by gw.catspoiler.org (8.13.3/8.13.3) with ESMTP id s7U7KkDk073975; Sat, 30 Aug 2014 00:20:50 -0700 (PDT) (envelope-from truckman@FreeBSD.org) Message-Id: <201408300720.s7U7KkDk073975@gw.catspoiler.org> Date: Sat, 30 Aug 2014 00:20:46 -0700 (PDT) From: Don Lewis Subject: Re: building an 8.4-STABLE i386 poudriere jail on an 10.0-STABLE amd64 host To: gjb@FreeBSD.org In-Reply-To: <20140830024255.GL1200@hub.FreeBSD.org> MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Cc: stable@FreeBSD.org, ian@FreeBSD.org X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Aug 2014 07:20:58 -0000 On 29 Aug, Glen Barber wrote: > On Fri, Aug 29, 2014 at 05:43:18PM -0700, Don Lewis wrote: >> I simplified things a bit and am now just doing normal crossbuilds. I'm >> still seeing core dumps, even on amd64 -> amd64 crossbuilds. I'm not >> seeing a reason, though ... >> >> [gdb backtrace] >> >> [...] >> >> Looks like all the pointers are OK, so why the SIGBUS? >> > > Pardon any incoherency in what follows, because it is mostly based off > smashed-together recollection from a text file of notes with the 8.x > (and prior) cross builds. > > The TL;DR is, there's probably not much information I can provide that > will be useful to you in debugging this. Sorry. :( > > (And Warner, if you're listening, I know your stance on the topic, so > no need to re-enlighten me.) :-) > > While it may sounds like I'm going in a different direction than what > you're try to solve, there actually is a reason for it, and a possible > solution for your case. > > The issues you are running into are 80% of the reason that after 8.4 was > released, there were no 8.4-STABLE snapshots produced. The other 20% is > split up with 10% because of how the release builds for 8.x and earlier > work, and 10% because it was not reasonable to implement "special" code > in the scripts I use for a single branch. If this was a few years ago, > and 7.x (and possibly 6.x) was still supported, I may have considered > it. That said, when I say "8.x" below, I really mean "8.x and prior > branches." > > My primary issue with how the 8.x release builds worked is that it would > use chroot(8) by default, specifically: > > 1) buildworld > 2) installworld DESTDIR=/blah > 3) chroot /blah make -C /usr/src buildworld buildkernel > 4) chroot /blah make -C /usr/src/release release I did a lot of release builds in the days of 4.x and that sounds familiar ... > The 9.x and later release builds do not do this. You specify the target > directory for the resulting build, and a bunch of magic happens, and > when it's done, you have all the things to install FreeBSD. The major > benefit to this is that it does *not* use chroot(8) by default. > > The 8.4-RELEASE was built on a machine running 9-STABLE, so make(1), > gcc(1), and so on were not that far diverged for there to be any real > issues (such as what you're seeing). Interestingly enough, make and clang don't seem to be an issue so far as I can tell. I do have the fmake port installed because poudriere told me that it was necessary, but if I just cd to the directory containing the source for 8-STABLE, "make buildworld" mostly seems to work. The incompatibility of the patch -b option was the first problem that I ran into, and that just required a few individual Makefile edits. The second problem was the lex upgrade during sometime after 10 was branched, I worked around that by making lex a bootstrap tool using the existing knob. Yacc was already a bootstrap tool. The final problem that is vexing me is that the cross-tool version of lint. The source for lint is basically unchanged between 8-STABLE and 10-STABLE. The only significant change that I noticed was a one line patch to lint1/tree.c (and lint1 is the SIGBUS victim) documented here: and fixed in r224702. I applied that patch and saw no change. If I run the cross version of lint with the -B option to get it to use the host version of lint1, then things work. It's just the cross version of lint1 that is broken. Both the host and cross versions of lint1 are compiled with clang, from nearly identical source code, though the cross version was built with a parser and lexer gernerated using the bootstrap versions of lex and yacc. What's really maddening is that I don't see anything wrong if I point gdb at the core file. The next thing I'm going to try is to try to run lint1 under gdb and single step it. That's tricky because lint1 is run from lint and I need to figure out how to set up its arguments and input file. Oh, and I discovered that neither "truss -fa" nor ktrace output anything for exec*() calls. > The snapshot builder in use now, at that time was running 10-CURRENT. > Several "hacks" were in place, such as WITH_GNUCXX (which somehow turned > into 4 or more src.conf(5) entries by now), which would allow building > 8.x on 10-CURRENT to somewhat work. That did not last long, though. Oh yeah ... I stumbled across that problem as well. I think I only needed WITH_GCC and WITH_GNUCXX. > I think it was around when bmake became the default in -CURRENT that > I stopped spending time trying to build 8.x snapshots -- at that point, > the time spent debugging the build failures and updating the scripts to > "fix" the build became increasingly less beneficial. > > Now, having said all that, I have a possible solution that, quite > honestly, may or may not work (I have no idea at this point). > > You can grab the 8.4-RELEASE distribution set from FTP, and extract it > into a scratch directory, check out the stable/8/ tree into the scratch > directory usr/src/, then do: > > # chroot /scratch make -C /usr/src buildworld [...] > > That would at least provide you with the binary bits you want for your > package builds. I've actually got a couple of 8-STABLE boxes here already. They are just underpowered for mass building packages, which is why I'm not just running "poudriere bulk" on one of them. The other missing piece is going from there to a working poudriere jail while not using the "jail -c" command. Hmn ... maybe I could build the jail on one of my 8.4-STABLE machines and just copy the jail bits over ... > I *think* this is somewhat what the cluster package builders do, but > then again, I do not know off-hand if packages for 8.x are build for > pkg(8) - they may be only pkg_install(1). I think they are building pkg(8) packages, but they just use 8.4-RELEASE, plus security updates I imagine. I've been able to do that here for testing individual package builds on 8.x. > Again, sorry that I do not provide a "just do $this" kind of magic wand > in this case, especially since I've been in the same painful situation > you are in. But I hope at least some of this information is useful to > you. > > Glen >