From owner-svn-src-all@FreeBSD.ORG Tue May 27 20:31:31 2014 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F1C548EE for ; Tue, 27 May 2014 20:31:31 +0000 (UTC) Received: from mail-ie0-f176.google.com (mail-ie0-f176.google.com [209.85.223.176]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A9EE422F0 for ; Tue, 27 May 2014 20:31:31 +0000 (UTC) Received: by mail-ie0-f176.google.com with SMTP id rl12so9427781iec.35 for ; Tue, 27 May 2014 13:31:30 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:message-id:references:to; bh=eZGeduO6oLH3YteEL0FeI/TNhnA16HRQM2pu+HgC3jM=; b=RyDM9HWHFlr+XaRccJPmzsbTsDTPi3bwBGrYg64VA6uzyC/vz8v6UzKizGWlUHBa1G WJVzwCL6tvrGD5CHw1iN5eUGkeJd91S44B7Q87FTg3tBEuMc+GOCaOop8n/v/N5G7Kit rFk5tIWzimJ9283pnPlbral3QW26TUkpUOHdlvRlhcbru4o7zBE0ySlWOpkw2AQQl+1C im5Nd2WIThWZsiAjCrMWmBAWtvX7OUPu9Lal9I/iWLFvhu+kYgm+VwDrRWcSfaZj41P4 nr0O5mzMqIVVslByqexOcj/yA2RHypj7yZL8LRHDKLLzMGldy8/OqppDL5cLiezY/iOa /DzQ== X-Gm-Message-State: ALoCoQmA0JhOjcL8CfXU64qssFuh8kl2zZYTAaQL/CHha1w9svxQMj0SS5E93tWflGMtFfZRQ1Ma X-Received: by 10.50.20.137 with SMTP id n9mr25877046ige.15.1401222689923; Tue, 27 May 2014 13:31:29 -0700 (PDT) Received: from [10.0.0.119] (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPSA id ng17sm7380488igb.13.2014.05.27.13.31.28 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 27 May 2014 13:31:29 -0700 (PDT) Sender: Warner Losh Content-Type: multipart/signed; boundary="Apple-Mail=_F9B225B6-1C99-4233-A917-99CE5519C109"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.2\)) Subject: Re: svn commit: r266553 - head/release/scripts From: Warner Losh In-Reply-To: <20140527214038.17d00369@kalimero.tijl.coosemans.org> Date: Tue, 27 May 2014 14:31:44 -0600 Message-Id: <13EB325C-3882-46AA-9B17-3BF19997C978@bsdimp.com> References: <201405221922.s4MJM4Y9025265@svn.freebsd.org> <20140523153619.GF72340@ivaldir.etoilebsd.net> <537F6EBC.3080008@freebsd.org> <20140523162020.GG72340@ivaldir.etoilebsd.net> <20140524165940.3c687553@kalimero.tijl.coosemans.org> <5380C311.60201@freebsd.org> <20140524185345.263f230d@kalimero.tijl.coosemans.org> <1400955835.1152.323.camel@revolution.hippie.lan> <5380EBA8.1030200@freebsd.org> <20140525011307.142b41ab@kalimero.tijl.coosemans.org> <3CCAFAD3-FABE-40EF-ABF9-815FE5826349@bsdimp.com> <9FE34CE4-C71F-4806-9EF6-30CB1051C62F@bsdimp.com> <20140526113502.239db74d@kalimero.tijl.coosemans.org> <5383522F.30108@freebsd.org> <20140527001811.3e9d3e8d@kalimero.tijl.coosemans.org> <05D1A11D-5985-42EA-84AD-209A8B51D391@bsdimp.com> <20140527093633.0a922e13@kalimero.tijl.coosemans.org> <85FABD2B-81BB-4E1A-B61E-4216A144A9DB@bsdimp.com> <20140527214038.17d00369@kalimero.tijl.coosemans.org> To: Tijl Coosemans X-Mailer: Apple Mail (2.1878.2) Cc: Baptiste Daroussin , src-committers@freebsd.org, Ian Lepore , svn-src-all@freebsd.org, Glen Barber , Nathan Whitehorn , svn-src-head@freebsd.org X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 May 2014 20:31:32 -0000 --Apple-Mail=_F9B225B6-1C99-4233-A917-99CE5519C109 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On May 27, 2014, at 1:40 PM, Tijl Coosemans wrote: > On Tue, 27 May 2014 07:18:06 -0600 Warner Losh wrote: >> On May 27, 2014, at 1:36 AM, Tijl Coosemans wrote: >>> On Mon, 26 May 2014 16:31:21 -0600 Warner Losh wrote: >>>> On May 26, 2014, at 4:18 PM, Tijl Coosemans = wrote:=20 >>>>> On Mon, 26 May 2014 09:53:57 -0600 Warner Losh wrote: >>>>>> On May 26, 2014, at 8:39 AM, Nathan Whitehorn = wrote: >>>>>>> On 05/26/14 02:35, Tijl Coosemans wrote: >>>>>>>> I suppose you could replace the "x86" in the pkg scheme with = i386/amd64, >>>>>>>> but then you'd still be talking about i386:32, amd64:32 and = amd64:64 >>>>>>>> instead of x86:32, x86:x32 and x86:64. =20 >>>>>>=20 >>>>>> I suppose you could replace these by ?i386?, ?x32? (or = ?amd64x32?) and >>>>>> ?amd64? respectively. >>>>>=20 >>>>> So you're on an amd64 or mips64 system (as indicated by uname) but = you >>>>> want to use the 32-bit package if possible. How does your script = know >>>>> about the magic "x32", "amd64x32" or "mipsn32" strings? Wouldn't = it be >>>>> easier if you could just use "`uname -p`:32?? >>>>=20 >>>> Oh give me a break. You know it because you know you are building = for >>>> mipsn32 because that?s what you?ve set MACHINE_ARCH or TARGET_ARCH = to, >>>=20 >>> No, MACHINE_ARCH or TARGET_ARCH is "amd64" or "mips64". You are = building >>> the 64-bit OS and then decide separately per package whether you = want the >>> ILP32 one or the LP64 one. >>=20 >> I think I understand why we?re talking past each other. This bit is = wrong. >> The LP64 one has a MACHINE_ARCH of ?amd64? or ?mips64?. The ILP32 = one will >> have MACHINE_ARCH of ?i386? or ?mips?. The weird ones (ILP32 with = 64-bit >> registers) will have a different MACHINE_ARCH of ?x32? or ?mipsn32?. >> Selection can be done on a case by case basis, but this will be = validated >> against the supported_abis sysctl. There?s no need to have different = names >> here, the current standard ones do just fine, are completely = sufficient and >> all inclusive. This is rather by definition, and your understanding = of the >> definition sounds flawed. >>=20 >> While we have limited support for building 32-bit binaries, it is to = build >> 32-bit binaries for a different MACHINE_ARCH. -m32 on amd64 creates = i386 >> binaries, not amd64:32 binaries. -m32 on powerpc64 creates powerpc = binaries, >> not powerpc64:32 binaries. On mips, -mabi-n32 (I think the option is) = is >> required to create the mipsn32 binaries. In every single one of these = cases, >> there exists a MACHINE_ARCH that completely describes the binary. >=20 > amd64:32 is provided by the -mx32 compiler flag. True, but not relevant. >> So I?m still waiting for a use case that requires the new names. One = has >> not been articulated, and I don?t think one actually exists. >=20 > Imagine you've built a system with MACHINE_ARCH amd64 and one with > MACHINE_ARCH mips64. Now you want to populate these systems with a = list > of packages for which you wrote a script. These systems each support = 2 > ABIs: a native 64-bit one (amd64 and mips64) and a native 32-bit one = (x32 > and mipsn32). Both are native in the sense that they make full use of = the > instruction set. This is not like i386 on amd64 or mips o32 on mips64 > because those are more like compat shims that operate under special > (crippled) cpu modes that nobody uses unless they're stuck with old = code. > Both our ABIs on the other hand are native and equally valid and which = one > to use for a particular package depends entirely on the use case. If = your > use case requires more than 4G of address space you'll have to use the > 64-bit package, otherwise you can use the 32-bit package which, = depending > on how pointer heavy the code is, may give a performance benefit. You > make this choice for each of the packages in your list and add that > information to your script. Long hypothetical, but so what? In such a case, you=92d pick one of two = different MACHINE_ARCH values depending on the package. This this is a = fairly atypical use case, it would not be unreasonable for the person = wanting to do this to know the proper companion ABI. In both cases, you = have the choice of two other values to use. Which one you use will = depend on a variety of factors, and what might be right for one = application may be wrong for others. And in both cases, there=92s actually two choices: for amd64, you=92d = have i386 and x32. Both of these are fine choices, and it would depend = on the workload which one is better (i386 has better toolchain support = and maturity, x32 offers some interesting theoretical wins, but doesn=92t = have the same maturity). Same with mips64, you=92d have two choices as = well. In both cases, you can=92t just take uname/MACHINE_ARCH and slap = :32 on the end. > Now let's work with Nathan's patch which uses the following string to > identify the pkg repository to fetch packages from: > `uname -s`:`uname -r | cut -f 1 -d .`:`uname -p` >=20 > On our two systems that would be FreeBSD:11:amd64 and = FreeBSD:11:mips64. > Now if your script has to install the 32-bit version of a package how > can it go from those two strings to FreeBSD:11:x32 or = FreeBSD:11:mipsn32 > without a lookup table? You couldn=92t. Which is the whole reason I want to have them have a = standard name so you don=92t need the lookup table for the common case. = This is an =93off in the weeds=94 case, and optimizing for it doesn=92t = make sense. Especially because in each case, you have two different = 32-bit ABIs to choose from. You=92d have to have some kind of table in = either case. Also, the proper name for n32, in your current system, is = mips:32:n32, which (a) is wrong and (b) isn=92t regular. > I say, you can more easily indicate whether you > want the 32-bit or 64-bit package by appending :32 or :64 to the = original > strings, so FreeBSD:11:amd64:32 and FreeBSD:11:mips64:32. Except there=92s no such thing as mips64:32 in the current system. = There=92s two different ABIs that could mean. It could be o32 or n32, = with the same sort of trade offs. There=92d need to be a person in the = loop to know, so there=92s already a need to have special knowledge. You = can=92t get there by just adding :32 to the existing thing... > Like I said in a reply to Nathan's patch, pkg could default to :32 or = :64 > for every arch so it can be left out in many cases. FreeBSD:11:i386 > would then be equivalent to FreeBSD:11:i386:32, FreeBSD:11:amd64 to > FreeBSD:11:amd64:64, etc. I don=92t see what value that adds to have the extra :32 or :64. = Nathan=92s patches make it possible to have an automated build system = with the typical use case (I want to build all my binaries, packages, = etc the same). But adding a :32 isn=92t going to even work for your = hypothetical example because people actually building such systems will = need to specify which 32-bit ABI they want to use anyway. And the = degenerate case of :32 just doesn=92t work with mips=85=20 Consider too that we=92re planning a new i386 ABI, which is identical to = the current i386 ABI, except time_t is 64 bit. We=92ve penciled in the = name i386t64 for this ABI/MACHINE_ARCH. This will be much less painful, = it is thought, than finding all the current places that take time_t as = an arg and shimming=85 So if we were to do that, then your current = scheme wouldn=92t be able to encompass that eventuality and we=92d be = back to this argument... > This also does not preclude the existence of a mipsn32 MACHINE_ARCH in > case you want to build a pure mipsn32 system (including the kernel), = but > I wonder how many people would use that if it is possible to have a > mips64 system run n32 binaries. I don't think many people would run a > pure x32 system (with x32 kernel) so I don't see the need to have an > "x32" value for MACHINE_ARCH (or TARGET_ARCH). Actually, you absolutely must have a x32 MACHINE_ARCH if you want to be = able to run x32 binaries. Otherwise, how are you going to build the = libraries that use that API? Our build system simply isn=92t setup to = build them any other way. Although the usr/lib32 stuff could grow extra = goo for that, you are still building them the same way you=92d build. We = have a stylized way to create a sys root, which is needed for the = compilers to work. We=92d likely have to grow better multi lib support = than we have now as well. You=92d need some way to identify these = binaries, segregate their ld.so, etc. Even if no kernel ever is linked = this way, it is still an absolute requirement. Warner --Apple-Mail=_F9B225B6-1C99-4233-A917-99CE5519C109 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJThPYwAAoJEGwc0Sh9sBEA9EAQANrGMeeK9cJHf4BLpU61flVT HmdMcrdFzKWzEUgkzGsPgOX13a83FnNAGrROdgGANKxUqS21g0VdsqhOgLgyHS3d u6gKCJtxfe4JA6bTO8zO+uiUqiyAv3+4qrZEbH3g5kCMjM+TQjj5na4+/odq0dYx 0Ud3Ff7nu7uMlK6SojoHYU78Nvb7M3oLmvKvc0clEX9RS47hdmm8CJOjO8eMv3Gw qFrHjJwo+S4pxxuYzUOG4CHiKckJ/hlkChkWV0R0TLfxwpNZTocwpiM9rPxPfMzH qsw0Ifu9xhN9lBbXsdhKR5lAFfqtPLFh/Jb52aFl4bDQMtYHb6PG9cJDuFezFOeY L6Ga3SPXvr7tNCpUETFwmhDgxruDedsNLyfEGsat5T4/64UnkwHVM1U/9ZcGwJQ+ 3YOb9oeJpf0Js8dw+734QDVU/RQqJYHyq7N3e+EwnF1lVcgnt+YAG2ISetj+NYN9 J33Hp2Txe7qonvGpI93WdvG3+oDhAYadIO5KoPXFbrZo2nv7aVVWnF0SXb1Wkt1R 2XENnDCvkg+TgCbmxQGJpsBa9gUkhnRTGIelwWDr843jE+umtbgEIQAzykTnt85d ajlBFjqlN3v3RipxOUTk978OGw+XCCZeh/gizFhf2Zf/XCVNPkKWEtEnLNfYLCnW eGizCocprieENZyv+bKw =YhAC -----END PGP SIGNATURE----- --Apple-Mail=_F9B225B6-1C99-4233-A917-99CE5519C109--