From owner-freebsd-current@FreeBSD.ORG Tue Jun 24 15:12:03 2014 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 83E63D64 for ; Tue, 24 Jun 2014 15:12:03 +0000 (UTC) Received: from mail-qa0-f50.google.com (mail-qa0-f50.google.com [209.85.216.50]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 344BF24C1 for ; Tue, 24 Jun 2014 15:12:02 +0000 (UTC) Received: by mail-qa0-f50.google.com with SMTP id m5so364946qaj.9 for ; Tue, 24 Jun 2014 08:11:56 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:message-id:references:to; bh=NlYTxtB9qOa7YiKp4XfeaKmaoFcHlHanXJKKeciW8jE=; b=Ga3QN+AD7eijyxSXTZeTftqNIJ9X+PhQvFYnvkPkUEdrQumFGMgIxMBkd8VNhti0cf Y+ahQit+M+7fHkMKVzmBuglpdBZnLXsdGZNU66sEVxZ5swXxhlNl+7oOPf7memszrmOR 9dQS4qmURD+gyP/jMH5KDYnI7HTl/IEt3mnGIj8oCf4gM/naWAMb6iVtgYppL1H8renr g30ig9QPyG9nNULswB6vNJMO0MGAN2DkniVyySpKJO9V/exyNtFNbSFt/BXQdPFWFWSQ nvh7byglseOsUApwuEur8g+/o/ICk+ce3Adacki6+zR4y+HrMnWvoe4nh36c79NN2xuH lbhw== X-Gm-Message-State: ALoCoQlH+r2rt4akBGPuOlmXi1gr2w3qcx6hMn2mMK3gc3/+YsPgnr0i8PHLz1MOLApkWRelzGRu X-Received: by 10.224.138.9 with SMTP id y9mr2694776qat.1.1403622716315; Tue, 24 Jun 2014 08:11:56 -0700 (PDT) Received: from [172.19.248.35] ([64.88.227.134]) by mx.google.com with ESMTPSA id m1sm844259qaz.27.2014.06.24.08.11.51 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 24 Jun 2014 08:11:55 -0700 (PDT) Sender: Warner Losh Content-Type: multipart/signed; boundary="Apple-Mail=_D0D13F9E-7ACE-463F-8451-02869D8A848E"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.2\)) Subject: Re: Problems building FreeBSD 9.2 on FreeBSD 10 From: Warner Losh In-Reply-To: <20140624144347.GA48694@hub.FreeBSD.org> Date: Tue, 24 Jun 2014 09:11:42 -0600 Message-Id: References: <1ED3AC7E-0F74-46A7-BAAA-E30600DC23BB@bsdimp.com> <8CD24B0A-DF45-4437-BEBE-8C67B241DE93@bsdimp.com> <2063888D-CCAE-431B-A409-F17AA4422006@bsdimp.com> <20140624011251.GN1218@hub.FreeBSD.org> <60320DD3-56C4-4922-A537-FF94C392A45A@bsdimp.com> <20140624022410.GP1218@hub.FreeBSD.org> <3DC0E4E6-A4B5-43EE-BD21-38B68DAE42F1@bsdimp.com> <20140624144347.GA48694@hub.FreeBSD.org> To: Glen Barber X-Mailer: Apple Mail (2.1878.2) Cc: freebsd-current Current X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jun 2014 15:12:03 -0000 --Apple-Mail=_D0D13F9E-7ACE-463F-8451-02869D8A848E Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On Jun 24, 2014, at 8:43 AM, Glen Barber wrote: > Trimmed CC a bit. >=20 > On Mon, Jun 23, 2014 at 11:42:20PM -0600, Warner Losh wrote: >> On Jun 23, 2014, at 8:24 PM, Glen Barber wrote: >>> I sort of typed what I meant a bit backwards from what I intended to >>> write. What I meant (sort of) is, "I would like to discuss our = forward >>> thinking on backward-compatibility." >>>=20 >>> I fully understand forward-compatibility is not feasible. >>=20 >> We already build current back to the stable/8 branch. 7.x is no = longer feasible, supported or tested. stable/10 is the only one that is = required, but enough people use stable/9 machines it will work. stable/8 = has one customer that is keeping it going, so I suspect it will stop = working in the coming years, maybe before 11 is branched. >>=20 >=20 > To be clear, I am talking about the other direction. Meaning, being > able to "reliably" build N-2 from head/, without needing to do = silliness > like 'make make buildworld', or "not using -jN.=94 Yea, that=92s never been officially supported, but generally works. In = fact, in the past, you were required to have exactly the same version on = the host as you were building a release for to ensure nothing weird = happening. Having the full release process work across multiple major = versions and have it produce identical results to the exact version = built release is not well tested and caused all kinds of problems back = in the day. To make it work, you=92ll need to make it work. And you=92ll need to = keep it working as people break it. We focus on the project as =93I=92m = updating from version X to version Y, where X < Y=94 in all our make = infrastructure. While we could add bits where X > Y, and for the = release, there are likely several items that will need to be fixed to = get there. You are currently hitting this turbulence with cross-version = races in multiple job builds. By all means fix them, but since this is = an unusual use case (from a historical perspective), expect there to be = bumps, and expect there to need to be fixes to make it work (and also = from a historical perspective, expect people will break it innocently). >>> I hate to even suggest this, but the ports tree (ab)uses the notion = of >>> using the kern.osreldate for certain things. This, however, = requires >>> proper bumping of __FreeBSD_version when needed, and maintenance of = the >>> Makefiles for the kern.osreldate-specific things. >>=20 >> We already do that. It mostly works most of the time, so long as the = delta isn=92t too great, and we don=92t have high compiler/tools/make = velocity=85 Except we don=92t use the kernel version, but rather the = installed tools version as indicated by a .h file. That=92s more robust. >>=20 >=20 > True. Thank you for the sanity check. >=20 >>> The benefit to this is that it would help prevent pissing off ports >>> developers and make their lives a bit easier when userland / kernel >>> things change. It would, however, (expectedly) is that it would = force >>> src committers to do the right thing. Win-win, IMHO. >>=20 >> What should we do we aren=92t doing today? >>=20 >=20 > There have been a number of times where changes that should deem > a __FreeBSD_version bump necessary either 1) do not bump > __FreeBSD_version at all, or 2) bump __FreeBSD_version several days = (or > longer) later. So, we're left with a window of time where something = is > "different enough", but there is no corresponding version change to > reference. >=20 > This is somewhat tangential to my original annoyance here though. :) With -current, a few days is more than enough granularity. There are = bumps, and this is one of them. >>> Personally, and no I won't discuss more on this, I'm in the camp of = "I >>> don't really see clang as a feature." It caused our ports = developers >>> and maintainers a mountain of headache to convert to the "invisibly = new >>> great thing", it increases our overall buildworld by a = non-insignificant >>> amount of time, and it has personally caused me headaches (still >>> ongoing) trying to figure out what the correct incantation of evil = to >>> wish over the cauldron to get BeagleBone images to build. (They're >>> failing because gcc is not being installed on both head/ and = stable/10/, >>> and despite the game of "musical KNOBS" I've been playing over the = past >>> few days, I'm running out of hair to pull out of my head.) >>=20 >> Yea, if you are using crochet, that=92s because crochet uses xdev = rather than a ports compiler (which in all fairness didn=92t exist when = it started) to build u-boot, which basically requires gcc. >>=20 >> The compiler rework in head is still a work in progress. What=92s = there now is better than before, but still isn=92t quite right. I do = plan on fixing that before summer is out. >>=20 >=20 > It isn't just head that is a problem with crochet, though. stable/10 > has been a problem since, as far as I can tell, roughly early May. Building release 9 on 10 falls under the X > Y category above. If if = breaks, and you want it to work, you gotta fix it. If there=92s several = somebodies that want it working, they gotta keep it working and fix what = breaks. Since this scenario is quite a bit less supported and has = received no love traditionally, it will likely take a while before you = can count on it once you=92ve mopped up the problems and convinced prime = movers to spend the extra time it takes to make this work. Supporting = all the normal use cases is hard enough, but adding this to the mix will = add lots of extra test time, and given how hard these problems have been = to nail, lots of extra debugging and fixing time to the cycle that might = be hard to enforce in a volunteer organization. >>>> But 9.2 will never build on head because it is broken with bmake, = which is now standard for head. Since 9.2 cannot be changed, and since = we=92ve removed (or nearly) fmake in current, chances are quite good it = will never build on head again without some special handling. >>>>=20 >>>> In summary, good luck! there=92s a lot of use cases here, and it = will take time and effort of multiple people over the long haul to keep = it working. Best effort may be larger than you estimate=85 I won=92t = stand in your way, but I=92m afraid my time available to help is = limited. >>>>=20 >>>=20 >>> As Ozzy once sang: >>>=20 >>> "I'm just a dreamer >>> I dream my life away >>> I'm just a dreamer >>> Who dreams of better days=94 >>=20 >> Since I was commenting on the opposite problem of what you were = wanting comments on, my harshness is justified. >>=20 >=20 > My comment wasn't a comment on your comment. :-) >=20 >> What you want though, we largely already do, though maybe with a few = more warts than necessary (which we should try to fix). Most of the = warts are due to gcc/clang division being done badly and unsustainable = initially and the cleanup taking a bit of time, not specific version = issues. >>=20 >> Back to your basic point, the issue becomes a testability one: not = all committers can reasonable be expected to have 8 or 9 systems to test = every change. Having a 10.x system to test changes is a bit of a stretch = as it is, but it is the official policy that many folks play fast and = loose with the rules because they haven=92t been burned too often by it=85= VMs, Jails, etc of various flavors can help, but some info does leak = through (mostly the info leaks are bugs or kludges that well meaning = people shouldn=92t have done given the historical knowledge we have = about the ill effects of certain ways to do conditional compilation). >>=20 >=20 > To be fair, we do have reference machines in the cluster running = head/, > stable/10, and stable/9. True, but I already have jails with these environments locally. It = doesn=92t magically give me the time to run the tests, nor find the time = to fix subtle problems that crop up. Plus, my work flow needs things = like hg and hgsubversion installed on top of the normal tools, which = historically has been at best spotty coverage on the project machines. = And despite using FreeBSD for two decades and being a somewhat active = developer, I=92ve never once typed =91make release=92. I don=92t mean to be difficult or unsympathetic, I=92m just trying to = paint a realistic picture of the challenges in expanding the current = support in builds to include the new things you want to include. Warner --Apple-Mail=_D0D13F9E-7ACE-463F-8451-02869D8A848E Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJTqZUuAAoJEGwc0Sh9sBEATJsP/3jaSaouzKxOAsXANiAHcwqS bP4gqIA0kBGVGeRREJUsG6IT5OzwhP2ECqr1PkikjB8o6nosCQSxnZewS7RFPiwd ZbNQ4OtbCltN80RQwoELDJGTwVHZpldDpWDRKsiy4M6an7ekxKTDCaTZTEU98apA 5mUUrWnHuRMkzYNhW42r06EQIGnvlkspDSvzNxceqD6YcZRJ3mv5R3UL/n0hsKEE aJbXqnigcvv+6E/WF9N/vgr+DPOpZsechCSRFwBhyWt+K6KhKHvbx1K7uRU84DY+ 4yMlJjUmg8tLtgxy8Zp5/mg6H05QJH/xH7k09HEiDHd69yGOIKKuhINfmtBmI1f5 Xun4YxbLZE7c5Hka5sQvrZlLXAmC2UuJEsdTIEBoL6QBaEwhad3gAOZrMkr419cZ 7RYpc4QuIyQi9VVjH2gj/4XIntmNVOxR4ySvU+Q/CL/zibNpKZNhaPFilv5VxC+u RXLCQvnul7vkUsW6/H6czdeS92YRSJGzz0uedyc6BySr/xyP9SKS06Xfv6wZzVBd slYY8+MwjfvkwfJ52FhpSjtlf2mlFJywMBhlKPBS2QgZUEjMm9DAPWX1joXvO+xS cGqtsBZXWOP7SNqeN49WU2oz2jKOCqMwyroKcqSeiYu224Dxs/DtJxdOeNNsWyxg Up0IpS6vn5B43oUfTCF7 =s7iD -----END PGP SIGNATURE----- --Apple-Mail=_D0D13F9E-7ACE-463F-8451-02869D8A848E--