Date: Wed, 28 May 2014 08:26:58 -0600 From: Warner Losh <imp@bsdimp.com> To: Tijl Coosemans <tijl@freebsd.org> Cc: Baptiste Daroussin <bapt@freebsd.org>, src-committers@freebsd.org, Ian Lepore <ian@freebsd.org>, svn-src-all@freebsd.org, Glen Barber <gjb@freebsd.org>, Nathan Whitehorn <nwhitehorn@freebsd.org>, svn-src-head@freebsd.org Subject: Re: svn commit: r266553 - head/release/scripts Message-ID: <5E038619-5921-4B7A-A4EE-D1E83614934B@bsdimp.com> In-Reply-To: <20140528125027.6d0cc4fb@kalimero.tijl.coosemans.org> References: <201405221922.s4MJM4Y9025265@svn.freebsd.org> <20140523162020.GG72340@ivaldir.etoilebsd.net> <C5A59513-AF58-4749-BCD7-F54BB6F56E90@gmail.com> <20140524165940.3c687553@kalimero.tijl.coosemans.org> <5380C311.60201@freebsd.org> <20140524185345.263f230d@kalimero.tijl.coosemans.org> <1400955835.1152.323.camel@revolution.hippie.lan> <5380EBA8.1030200@freebsd.org> <20140525011307.142b41ab@kalimero.tijl.coosemans.org> <3CCAFAD3-FABE-40EF-ABF9-815FE5826349@bsdimp.com> <9FE34CE4-C71F-4806-9EF6-30CB1051C62F@bsdimp.com> <20140526113502.239db74d@kalimero.tijl.coosemans.org> <5383522F.30108@freebsd.org> <DAD3E386-6555-4C43-9BBA-F3BFD28CC19B@bsdimp.com> <20140527001811.3e9d3e8d@kalimero.tijl.coosemans.org> <05D1A11D-5985-42EA-84AD-209A8B51D391@bsdimp.com> <20140527093633.0a922e13@kalimero.tijl.coosemans.org> <85FABD2B-81BB-4E1A-B61E-4216A144A9DB@bsdimp.com> <20140527214038.17d00369@kalimero.tijl.coosemans.org> <13EB325C-3882-46AA-9B17-3BF19997C978@bsdimp.com> <20140528125027.6d0cc4fb@kalimero.tijl.coosemans.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--Apple-Mail=_00A58E78-DF71-466C-86E8-77A748FE57EB Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On May 28, 2014, at 4:50 AM, Tijl Coosemans <tijl@freebsd.org> wrote: > On Tue, 27 May 2014 14:31:44 -0600 Warner Losh wrote: >> On May 27, 2014, at 1:40 PM, Tijl Coosemans <tijl@FreeBSD.org> wrote: >>> On Tue, 27 May 2014 07:18:06 -0600 Warner Losh wrote: >>>> On May 27, 2014, at 1:36 AM, Tijl Coosemans <tijl@FreeBSD.org> = wrote: >>>>> On Mon, 26 May 2014 16:31:21 -0600 Warner Losh wrote: >>>> So I?m still waiting for a use case that requires the new names. = One has >>>> not been articulated, and I don?t think one actually exists. >>>=20 >>> Imagine you've built a system with MACHINE_ARCH amd64 and one with >>> MACHINE_ARCH mips64. Now you want to populate these systems with a = list >>> of packages for which you wrote a script. These systems each = support 2 >>> ABIs: a native 64-bit one (amd64 and mips64) and a native 32-bit one = (x32 >>> and mipsn32). Both are native in the sense that they make full use = of the >>> instruction set. This is not like i386 on amd64 or mips o32 on = mips64 >>> because those are more like compat shims that operate under special >>> (crippled) cpu modes that nobody uses unless they're stuck with old = code. >>> Both our ABIs on the other hand are native and equally valid and = which one >>> to use for a particular package depends entirely on the use case. = If your >>> use case requires more than 4G of address space you'll have to use = the >>> 64-bit package, otherwise you can use the 32-bit package which, = depending >>> on how pointer heavy the code is, may give a performance benefit. = You >>> make this choice for each of the packages in your list and add that >>> information to your script. >>=20 >> Long hypothetical, but so what? In such a case, you=92d pick one of = two >> different MACHINE_ARCH values depending on the package. This this is = a >> fairly atypical use case, it would not be unreasonable for the person >> wanting to do this to know the proper companion ABI. In both cases, = you >> have the choice of two other values to use. Which one you use will = depend >> on a variety of factors, and what might be right for one application = may >> be wrong for others. >>=20 >> And in both cases, there=92s actually two choices: for amd64, you=92d = have >> i386 and x32. Both of these are fine choices, and it would depend on = the >> workload which one is better (i386 has better toolchain support and >> maturity, x32 offers some interesting theoretical wins, but doesn=92t = have >> the same maturity). Same with mips64, you=92d have two choices as = well. In >> both cases, you can=92t just take uname/MACHINE_ARCH and slap :32 on = the end. >=20 > As far as I can see nobody uses i386 on amd64 except in cases where = you > know you're dealing with i386 like an i386 jail, or you need to run an = old > i386 binary that you're stuck with, or with compat shims like Wine. = The > choice of i386 is not comparable with the generic choice between ILP32 = or > LP64 on a 64-bit instruction set, which is why I'm ok with changing = the > "x86" in the pkg scheme to either "i386" or "amd64" like you are = proposing. > If pkg would keep using instruction set families like "x86" or "mips" = then > you could distinguish between old 32-bit, new 32-bit and new 64-bit = using > 3 generic suffixes to avoid lookup tables. The pattern of an old = 32-bit > instruction set that got extended to 64-bit on which you can define a = new > 32-bit ABI is common to many instruction set families, but I'm ok with > old 32-bit and new 64-bit having separate MACHINE_ARCH values. OK. So we use MACHINE_ARCH for the pkg scheme. I=92m glad we agree on = this point. That will make adding support for pkg repos and multiple = machine builds to nanobsd much easier. >>> Now let's work with Nathan's patch which uses the following string = to >>> identify the pkg repository to fetch packages from: >>> `uname -s`:`uname -r | cut -f 1 -d .`:`uname -p` >>>=20 >>> On our two systems that would be FreeBSD:11:amd64 and = FreeBSD:11:mips64. >>> Now if your script has to install the 32-bit version of a package = how >>> can it go from those two strings to FreeBSD:11:x32 or = FreeBSD:11:mipsn32 >>> without a lookup table? >>=20 >> You couldn=92t. Which is the whole reason I want to have them have a >> standard name so you don=92t need the lookup table for the common = case. >> This is an =93off in the weeds=94 case, and optimizing for it doesn=92t= make >> sense. Especially because in each case, you have two different 32-bit >> ABIs to choose from. You=92d have to have some kind of table in = either >> case. Also, the proper name for n32, in your current system, is >> mips:32:n32, which (a) is wrong and (b) isn=92t regular. >=20 > Well, it may not make sense to you and be off in the weeds, but it = does > to me. Very few processes actually need more than 4G of virtual = address > space. I cannot immediately find anything in /bin or /usr/bin for > instance (maybe clang when compiling a very large C++ file?). >=20 > I'm not sure where you're getting mips:32:n32 from. We're talking in = the > context of Nathan's patch here. In the context of Nathan=92s patch, mips:32:n32 is the =93old=94 name = that is an alias for mipsn32. There=92s no =93mips64:32=94 defined at = all, so that pattern breaks. That was my specific point. >>> I say, you can more easily indicate whether you >>> want the 32-bit or 64-bit package by appending :32 or :64 to the = original >>> strings, so FreeBSD:11:amd64:32 and FreeBSD:11:mips64:32. >>=20 >> Except there=92s no such thing as mips64:32 in the current system. = There=92s >> two different ABIs that could mean. It could be o32 or n32, with the = same >> sort of trade offs. There=92d need to be a person in the loop to = know, so >> there=92s already a need to have special knowledge. You can=92t get = there by >> just adding :32 to the existing thing... >=20 > Yes, I am assuming that o32 on mips64 is like i386 on amd64, i.e. only = used > in cases where you know you're dealing with mips o32, and that = normally n32 > is the 32-bit ABI of choice on mips64 because it provides better = performance. That=92s not a good assumption. In fact, it is a terrible assumption = given the relative maturity of the o32 and n32 implementations on = FreeBSD mips. o32 is extremely well tested and works fairly well. n32 = works, mostly, but has many edge cases that seem a little off that = haven=92t been fully shaken out. >>> Like I said in a reply to Nathan's patch, pkg could default to :32 = or :64 >>> for every arch so it can be left out in many cases. FreeBSD:11:i386 >>> would then be equivalent to FreeBSD:11:i386:32, FreeBSD:11:amd64 to >>> FreeBSD:11:amd64:64, etc. >>=20 >> I don=92t see what value that adds to have the extra :32 or :64. = Nathan=92s >> patches make it possible to have an automated build system with the >> typical use case (I want to build all my binaries, packages, etc the >> same). But adding a :32 isn=92t going to even work for your = hypothetical >> example because people actually building such systems will need to >> specify which 32-bit ABI they want to use anyway. And the degenerate = case >> of :32 just doesn=92t work with mips=85 >>=20 >> Consider too that we=92re planning a new i386 ABI, which is identical = to >> the current i386 ABI, except time_t is 64 bit. We=92ve penciled in = the >> name i386t64 for this ABI/MACHINE_ARCH. This will be much less = painful, >> it is thought, than finding all the current places that take time_t = as >> an arg and shimming=85 So if we were to do that, then your current = scheme >> wouldn=92t be able to encompass that eventuality and we=92d be back = to this >> argument... >=20 > Nobody will use this new ABI on amd64 when x32 is supported. X32 has = more > registers available and a far better calling convention. "Tools not policy=94 has been the mantra of the project. You are making = assumptions here that may not be correct today, and may change over time = and baking that into the tools. This seems unwise. >>> This also does not preclude the existence of a mipsn32 MACHINE_ARCH = in >>> case you want to build a pure mipsn32 system (including the kernel), = but >>> I wonder how many people would use that if it is possible to have a >>> mips64 system run n32 binaries. I don't think many people would run = a >>> pure x32 system (with x32 kernel) so I don't see the need to have an >>> "x32" value for MACHINE_ARCH (or TARGET_ARCH). >>=20 >> Actually, you absolutely must have a x32 MACHINE_ARCH if you want to = be >> able to run x32 binaries. Otherwise, how are you going to build the >> libraries that use that API? Our build system simply isn=92t setup to >> build them any other way. Although the usr/lib32 stuff could grow = extra >> goo for that, you are still building them the same way you=92d build. = We >> have a stylized way to create a sys root, which is needed for the >> compilers to work. We=92d likely have to grow better multi lib = support >> than we have now as well. You=92d need some way to identify these = binaries, >> segregate their ld.so, etc. Even if no kernel ever is linked this = way, >> it is still an absolute requirement. >=20 > I'm kind of hoping we can close this discussion now and agree to = disagree > on the points that we still disagree, so I don't really want to = discuss > our build system. We'd be off for another week. I understand why we > build compat32 libraries the way we do now. It was the only way at = the > time. In my opinion the choice between 32-bit or 64-bit should be = like > choosing between static or shared, pic or non-pic, or with or without > profiling. It's just a compiler flag while our current build system > treats it almost like a full blown cross compilation problem. Then we disagree on this point. However, the disagreement here is kinda = foundational: to build a set of libraries or sys root, you have to have = a MACHINE_ARCH to make it work. Even in our current system, we set = MACHINE_ARCH to i386 or powerpc when building the 32-bit binaries (note: = we don=92t do this for mips). This means that if we do grow x32 support, = we=92ll need to grow a MACHINE_ARCH for it. That=92s my point: all ABIs = have MACHINE_ARCH associated with them, and those are the names users = are used to specifying, and are the ones that are the most natural for = script writers to use. With nathan=92s patches, we=92re to the point = where those are used, though there=92s also the option of using the = non-standard names if you want (e.g. amd64:32 instead of x32). Believe me, I don=92t want to talk about the build system, but it is = relevant for this discussion because of the integration into the base = system. It can=92t just be ignored. Warner --Apple-Mail=_00A58E78-DF71-466C-86E8-77A748FE57EB Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJThfIyAAoJEGwc0Sh9sBEA0qYP/0oxa0vJSGzr9llmgCVKa4HB E3QsS+81NIKEcgXP0Ii0Jp5Jbl2JyfRQ1JW6LLg1+1S/qrTdLos/y9FCYTohIo9P VhQtGanOozBSrIDDLVsapKcZExfNH01UBXJL8dQD5Hm0mcMjez1wFR6Pg5NyHFYv m/4W2Cb8Rr6H6DENkazwrEmGZRwMj0ltOejcLc37xgBukYFltpFTVvY0GKATs6HB meWrR8koKG/KHNPqhYRkZqx2+yyLAspnKme2odkVt9ikw8cjDdqEgr2PEe19xIaq WOfIVr8wN+hTUWDoCm228y7ZxByBdawbaHHzZEYCyyHp1Wu3FbtqbmOAgo6jC/Hc 5pAtOM8zukYL9MQMCdTYlLh9NUYmfBFm3k6PjRyIP3qtTxVRDfZJVjvmMmMBTc9c LXiFdeq0/c/K/FGgoaA684zsdVA4U4h9CDiEpcYRzPyd7QKfCbOpw8VgWYRXrq3P Kpf0qkM/DxmBQwAr/3UoptcyBNjfEMK+Hq/bCCaROkO8ve0CQklzsgKkkMjHQCZl 8/HXBWfke02qXh6sFDFbgXIvMDkkRvA/9WFpeGm0RAabuoe0Xdtat+f+vuX4usUy WBqitIfqcFKiZSgDxut4pXPSIihLUN8ZAc+t5t9AX6N1XZs2LIXdwDdr00n++xv9 AFNSc9hA1e+MeUUyT5zg =Ih3d -----END PGP SIGNATURE----- --Apple-Mail=_00A58E78-DF71-466C-86E8-77A748FE57EB--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5E038619-5921-4B7A-A4EE-D1E83614934B>