From owner-freebsd-arm@freebsd.org Tue Jun 13 05:37:09 2017 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AD061D8E8FB for ; Tue, 13 Jun 2017 05:37:09 +0000 (UTC) (envelope-from russ.haley@gmail.com) Received: from mail-lf0-x22a.google.com (mail-lf0-x22a.google.com [IPv6:2a00:1450:4010:c07::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1E7DF6E430 for ; Tue, 13 Jun 2017 05:37:09 +0000 (UTC) (envelope-from russ.haley@gmail.com) Received: by mail-lf0-x22a.google.com with SMTP id p189so62138669lfe.2 for ; Mon, 12 Jun 2017 22:37:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=DzYlx/raTPkdCPbC3UPpiyzolvBC0r6asWEQwJqCxSQ=; b=J0NKh9FxfV7LUeImWYanKSnwxK9l1eal2/wrasM4DRH0gaUe24SQJfWwy5ynP8ZBBS v5xLagBxSUnednTN1FakHpYljOWh5tWObGaPxtFPcN8/rvq7Y4sDm81Vfurt/aAwCN6F 9bKy3NvgqDyJRzuqCPv/FN+QQfTRTTQxerIEBAnB8wQpEMwB7m6RIbacqQHEkGOaToC4 396FpEZD6scYA0B4bWPQon7bNDmUhbHSkxBarkY4kNFPiOgHC3kQzU1c1UzQ6ipxSUwq u9veAuDuteBUsWEyfskvm7IT0MDZGXkz9Y3Pj9uuChibHxICvqDwMoIg5dyeigViNGsM vApg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=DzYlx/raTPkdCPbC3UPpiyzolvBC0r6asWEQwJqCxSQ=; b=NxDmI42kDgLm0Ukjn84ai59q3JcLjQ3ElYxT7DBHReQXPHvjUH0b8BA+AMi11EtYkv XwO4zcS9tUPAxm/ZfMqIaQRCmGWG343FxlTiTpdHKFVDkZdD7tujbV67KkRN5DW2T/Is gbBRdyLKWvgN4Ns9H/kQ1QJYdnWmUVPRx3BKG7klqnqTPh049ot8IjMUll5K1we3dkqT T+ExUi56zgE5XZwpgd+EqOmx8lwmVl1OZOr5tCuFt7OML7wrAbpnoL2abz6x97U/MO7H WIreL5BxMa8eEAFsbYJB0rXzSLHilEJWbBYZIGeKT1KFdwvoWtUFI77b3ohVJeYVrAcV 5cew== X-Gm-Message-State: AODbwcAqGyOAW6xZoNGH5QIMNgmlepJA6nP3n7r5jnW9IluFZ1LitYVD T4ULVLha+Hzw2nP8QHYzjA338/NvJw== X-Received: by 10.46.13.2 with SMTP id 2mr18719485ljn.93.1497332226735; Mon, 12 Jun 2017 22:37:06 -0700 (PDT) MIME-Version: 1.0 Received: by 10.46.82.196 with HTTP; Mon, 12 Jun 2017 22:37:05 -0700 (PDT) In-Reply-To: References: <20170612152808.6094931.74364.27128@gmail.com> <2A90A527-7DCA-4442-9322-0EA96236583C@dsl-only.net> <3CC8DE8A-CCF2-4856-A43E-6B259BDE8B2C@dsl-only.net> From: Russell Haley Date: Mon, 12 Jun 2017 22:37:05 -0700 Message-ID: Subject: Re: Creating armv7 MACHINE_ARCH To: Mark Millard Cc: "freebsd-arm@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Jun 2017 05:37:09 -0000 Thanks Mark! I'll chew on that for a couple of days. :D Russ On Mon, Jun 12, 2017 at 7:01 PM, Mark Millard wrote: > > On 2017-Jun-12, at 4:10 PM, Russell Haley wrote: > >> Okay, feel free to ignore me, I'm not going to get the time drill into >> the source code for my own questions so I don't expect anyone else >> too. However, I'll ask anyway. I'm too confused to try and inline >> these questions. Lets see if I understand: >> >> - armv7 does not support 64 bit instructions (according to Wikipedia? >> I claim no expertise.) > > It does not support AArch64 instructions but does support AArch32 > instructions (or is a good approximation). > > AArch32 was actually created later to be armv7-a like. > >> - FreeBSD has an armv6 "architecture" that is supports armv6 and armv7 >> on Pre-Cortex-A-53 processors that is not supported on A-53 through >> the emulated AArch32. > > There are actually architectures (by ARM's > definitions). . . > > ARMv6, ARMv6T2, ARMv6Z, ARMv6K, ARMV6-M > ARMv7-M, ARMv7E-M, ARMv7-R, ARMv7-A > ARMv8-A, ARMv8.1-A ARMv8.2-Am ARMV8.3-A, ARMv8-R, ARMv8-M > > I'm going to largely ignore much of > that variation in structure. > > ARM has continued to produce/make new variations for both > 32-bit and 64-bit in overlapping time frames. So the > "Pre-Cortex-A53" presumes to much about relative timing. > > Technically the old RPI2s (V1.1 and before) have > Cortex-A7's and the new V1.2's have Cortex-A53's. > RPI3 is also Cortex-A53 based. > > Cortex-A7 includes an implementation of armv7-A architecture > (but is not the only one). > Cortex-A53 includes an implementation of armv8-A architecture > (but is not the only one). > (Both include other things as well. And the System On A Chip has even > more stuff than the Cortex-A's in question.) > > Cortex-A's are specific CPUs/cores that are also > examples of the specific architecture(s) they implement. > > If a kernel supports armv7-a it can support a variety of > CPUs that all implement armv7-a architecture. It may not > support things in the CPUs that are beyond that > architecture. > > If a kernel supports armv8-a it can support a variety of > CPUs that all implement armv8-a architecture. It may not > support things in the CPUs that are beyond that > architecture. > > Note there are also issues of support of the System On > a Chip involved as well. More than ARM is involved > overall. > > > >> - Cortex A-53 can support armv8 (AArch64) and armv7 (AArch32) instructions > > As I remember armv8-a specifies: > > A) Optional support: AArch64 > (Cortex-A32 is ARMv8-A archtecture but only 32-bit) > (ARMv8-R architecture is always only 32-bit as I understand) > (ARMv8-M architecture is always only 32-bit as I understand) > B) Optional support: AArch32 > (Cortex-A35/53/57/72/73 are ARMv8-A and have both) > (Cortex-A55/75 are ARMV8.2-A architecture and have both) > (But variants can be produced that omit AArch32.) > C) Various things specific to armv8-a > might go beyond what AArch64 and > AArch32 specify. > > Note: AArch32 has 2 instruction sets > (A32/ARM and T32/Thumb), like armv7-a > does. AArch64 has A64. (I'm not giving > thumb version numbers here but there > are versioned vintages.) > > So 1 architecture (armv8-a) has at least > 3 instruction sets when both AArch64 > and AArch32 are implemented. > > AArch64 state: A64 > > AArch32 state: A32 and T32 > > There can be context switching between > these states as I understand. > > (I'll not get into the NEON distinctions > and such so the above is simplified.) > > armv8-a does not necessarily uniquely identify > the interrupt controller or its software > interface, as an example of variation that is > not an instruction set issue: register > interfaces to other hardware. ARM provides > GIC-400 and GIC-500 and others. But the > controller does not have to come from ARM. > > While Cortex-A53 has a default/reference > interrupt controller(s), it is possible to > build variations that use other ones as > I understand. > > Similar points go for Cortex-A7. > > The other parts of a System On a Chip are for non-arm > aspects and may have some mixed degree on commonality > independently of ARM. But the kernel may deal with > such specifics as well. > >> - The current proposal is to split the armv6 and armv7 into their own >> "architectures" > > ARM defines these as distinct architectures. > That is not a FreeBSD specific terminology. > But much of the armv6 is common with armv7. > > armv7 architecture has more than armv6 architecture does: > it is an update. Having armv7 implemented separately > in FreeBSD means more completely/correctly identifying > what is there compared to pretending it is an armv6. > >> FreeBSD has an Arm 64 bit kernel build. I don't see what the >> TARGET_ARCH settings in the wiki and know little about it, but will >> conjecture that it doesn't have a TARGET_ARCH=armv8 (please correct me >> if I'm wrong). > > FreeBSD has TARGET=arm64 TARGET_ARCH=aarch64 . That combination > is for armv8-A with AArch64 currently. Cortex-A32 is not > supported because it lacks AArch64. > > FreeBSD's TARGET=arm64 TARGET_ARCH=aarch64 may grow to span > armv8.1 or such as well for all I know. Cortex-A53 is one > example of an armv8 implementation that the kernel supports > (presuming certain interrupt controllers and such). There are > others. > >> What I was trying to ask was: is the kernel development moving in a >> direction that clearly indicates the differences in the instructions >> vs the architectures (or have I grossly simplified the problem)? Will >> it be possible to target a Cortex-A53 and build for 32 or 64 bit >> support? Or is this just to fix RPi? > > Instruction sets are part of an architecture but not all of it. > They are not independent of architecture. Talking of an overall > architecture includes covering the instruction set(s). One > architecture can have multiple instruction sets as part of it. > > TARGET=arm64 TARGET_ARCH=aarch64 may get a "lib32" (compat32) > implementation someday if someone is motivated to do the > work. That would enable AArch32 user processes. > > What exists now and will in the future is mostly tied to > choices of volunteers that say "I'm going to provide > " and then manage to complete whatever it was. > (Clearly a "we are going to" is also an option.) > > A similar points would go for TARGET_ARCH=armv7 supporting > something like Cortex-A53 in AArch32 mode (Raspian like). > > So at this point no one knows if or when as far as I know. > > armv6 and armv7 architectures are strongly overlapping for what > is in armv6. The GENERIC kernel for what is currently a > TARGET_ARCH=armv6 build actually builds for armv7 and will not > work on armv6. One has to use a different kernel configuration > than GENERIC for armv6 (such as the one for the older RPI vintage > that was armv6 based). > > Identifying an armv7-a build as a armv6 build has odd > consequences for software that tries to figure out what > can be depended on. > >> In terms of Raspbian, I had assumed they were just supporting Aarch32 >> across both processor models. Many of the drivers would be the same >> source if they share components so I would think it would be "simple". >> I didn't see anything in my brief look at it today to indicate >> otherwise. > > For Cortex-A53: Raspian supports just AArch32 state, > not AArch64 state for rpi2 V1.2 and RPI3. (I do not > know if early boot is temporarily AArch64 or not.) > > For Cortex-A7: Cortex-A7 predates AArch32 and is armv7-a > directly but AArch32 was designed to appear to be be > nearly the same as armv7-a. So Raspian is supporting > armv7-a directly for rpi2 V1.1 and V1.0. > > === > Mark Millard > markmi at dsl-only.net > > On Mon, Jun 12, 2017 at 2:07 PM, Warner Losh wrote: >> >> >> On Mon, Jun 12, 2017 at 2:35 PM, Mark Millard wrote: >>> >>> >>> On 2017-Jun-12, at 1:00 PM, Mark Millard wrote: >>> >>>> On Mon, Jun 12, 2017 at 1:00 PM, Mark Millard >>>> wrote: >>>>> On 2017-Jun-12, at 12:16 PM, Russell Haley >>>>> wrote: >>>>> >>>>>> On Mon, Jun 12, 2017 at 10:36 AM, Mark Millard >>>>> dsl-only.net> wrote: >>>>>>> >>>>>>> On 2017-Jun-12, at 8:39 AM, Warner Losh wrote: >>>>>>> >>>>>>>> . . . >>>>>>>> >>>>>>>> Plus, we aren't quite doing what Ian wanted. He wanted a full >>>>>>>> rename. The >>>>>>>> proposal on the able is to add an armv7 TARGET_ARCH in 12. Not to >>>>>>>> rename or >>>>>>>> remove armv6. Sadly, that will still be there since the RPI >>>>>>>> foundation >>>>>>>> keeps finding new ways to repackage the rpi into new boards that are >>>>>>>> just >>>>>>>> too cheap to ignore. >>>>>>> >>>>>>> On 2017-Jun-12, at 6:59 AM, Andrew Turner >>>>>>> wrote: >>>>>>> >>>>>>>> I like this. My understanding is adding armv7 would also fix many of >>>>>>>> the currently broken ports that assume they are being built for armv7 as >>>>>>>> many Linux distros target ARMv7+. >>>>>>>> >>>>>>>> It should also be noted the GENERIC kernel is likely to only ever >>>>>>>> target ARMv7+ even without an armv7 TARGET_ARCH. >>>>>>> >>>>>>> >>>>>>> Hopefully the choices related to TARGET and TARGET_ARCH >>>>>>> for armv7 end up identifying the context to port builds >>>>>>> so that many would just automatically do the right thing. >>>>>>> >>>>>>> >>>>>>> As for GENERIC: >>>>>>> >>>>>>> powerpc has. . . >>>>>>> >>>>>>> TARGET=powerpc TARGET_ARCH=powerpc and GENERIC >>>>>>> TARGET=powerpc TARGET_ARCH=powerpc64 and GENERIC64 >>>>>>> >>>>>>> So there is precedent for more than one GENERIC* >>>>>>> for a family, with which one being appropriate >>>>>>> being based on TARGET_ARCH. >>>>>>> >>>>>>> For powerpc TARGET=powerpc implicitly uses >>>>>>> TARGET_ARCH=powerpc when TARGET_ARCH is not >>>>>>> specified (if I remember right). Which should >>>>>>> be the default for armv6 vs. armv7 might go >>>>>>> the other direction (TARGET_ARCH=armv7 by >>>>>>> default). >>>>>>> >>>>>>> >>>>>>> Side note: >>>>>>> >>>>>>> A caution about talking about "rpi2" as >>>>>>> an example. . . >>>>>>> >>>>>>> Raspberry Pi 2 Model B V1.2 is Cortex-A53 based >>>>>>> (so arm64/aarch64). (A BCM2837, not a BCM2836.) >>>>>>> This dates about to something like 2014 based >>>>>>> on the pictures showing the (c) notice on the >>>>>>> boards. >>>>>>> >>>>>>> V1.1 and before were armv7 (BCM2836) based. >>>>>>> >>>>>>> Unless a kernel and world are made that can >>>>>>> also configure/handle a Cortex-A53 in a >>>>>>> armv7-like manor there will be two different >>>>>>> GENERIC builds in order to span the "rpi2" >>>>>>> family, based on just V1.2+ vs. V1.1 and >>>>>>> before. >>>>>>> >>>>>>> (A single, modern distribution of the official >>>>>>> Raspbian software for the rpi2 does support >>>>>>> all the V1.x boards if I understand right.) >>>>>> >>>>>> I am confused. I don't see any documentation about Raspbian supporting >>>>>> 64 bit? >>>>> >>>>> 64-bit Cortex-A53's can be configure to operate in a >>>>> 32-bit mode (AArch32). Raspian does that for RPI2 V1.2 >>>>> and for RPI3. >>>>> >>>>> Raspian does not (yet?) support a 64-bit mode (AArch64). >>>>> >>>>> The Cortex-A53 can support either. As I understand it >>>>> is possible for an OS to allow a user processes to be >>>>> one or the other, different processes using the different >>>>> modes. That does not mean that all operating systems >>>>> bother to. >>>>> >>>>> If I remember right the official Ubuntu for an ODroid-C2 >>>>> allows both AArch64 and AArch32 user programs (and >>>>> so processes, with shared library types tracking). >>>>> >>>>>> From Arm at >>>>>> https://www.arm.com/products/processors/cortex-a/cortex-a53-processor.php: >>>>>> "The Cortex-A53 supports the full ARMv8-A architecture. It not only >>>>>> runs 64-bit applications also seamlessly and efficiently runs legacy >>>>>> ARM 32-bit applications." >>>>>> >>>>>> I assume that means it handles armv7-A without issue? (In fact, many >>>>>> on this board know it does) >>>>> >>>>> I've not gone through the details but targeting AArch32 >>>>> probably means targeting a subset of armv7. Or may be >>>>> to support both requires targeting a common subset of both. >>>>> (My guess is that AArch32 is the definition of a common >>>>> subset for 32-bit, at least for user processes.) >>>>> >>>>> Raspian targets just AArch32 on armv7 and Cortex-A53 >>>>> (user space). (If I've got the definition of AArch32 >>>>> right: otherwise a common subset.) >>>>> >>>>> FreeBSD targets armv7 and AArch64 separately (via >>>>> separate GENERIC kernels). I'm not aware of FreeBSD >>>>> targeting AArch32 at all on cores capable of AArch64. >>>>> FreeBSD possibly does not restrict itself to AArch32 >>>>> (user processes) on armv7 if AArch32 is really a >>>>> subset. >>>> >>>> I thought all 64 bit Arm instructions are defined in armv8? >>> >>> (I assume you are not trying to indicate armv8.1, armv8.2 >>> and such. Cortex-A53 is older than those and so does not >>> have the newer things involved.) >>> >>> That Cortex-A53 allows armv8 64-bit arm code is not in >>> dispute. But the operating system in involved in setting >>> up what will actually work based on how it configures >>> things and operates. Much of this is the kernel. >> >> >> Correct. >> >>> >>> Cortex-A53 also supports AArch32, i.e., 32 bit instructions. >>> So that the 64-bit instructions all being there does not >>> of itself prevent using a 32-bit mode instead. >>> >>> (The kernel likely has to deal with more specifics of >>> processor variations than user code does not. My notes >>> are really about the user process model, not the all >>> the kernel details.) >>> >>> Raspian deals with armv7's that have no 64-bit support >>> and with Cortex-A53's that do. It presents a user-process >>> model that is 32-bit only, even on Cortex-A53's that allow >>> for 64-bit (but do not required user programs to be AArch64 >>> code). >>> >>> Ubuntu for ODroid-C2 does not deal with armv7's but does >>> allow both 64-bit (AArch64) and 32-bit (AArch32) user >>> processes as I remember, on its Cortex-A53's. >>> >>> FreeBSD armv7 does not support Cortext-A53 or anything >>> that allows 64-bit (that allows AArch64). This is a kernel >>> level issue. >> >> >> Not a hugely difficult issue to fix, but one nobody had fixed... >> >>> >>> FreeBSD aarch64 does not support having AArch32 user >>> processes. Nor does its kernel support processors that >>> do not support aarch64 (so it does not support armv7). >> >> >> Executing a 32-bit /bin/cat on aarch64 level support exists outside the >> tree, according to the hallway track at BSDcan, so it will only be a matter >> of time before compat32 exists there I think. >> >> There's a further complication is that the aarch32 unit of aarch64 >> processors is optional. Not all of them have it, so that can be a problem... >> IIRC, the early aarch64 targets didn't have this feature... >> >>> >>> These are simply examples of different choices about >>> what combinations of the technical possibilities to >>> put effort into supporting in the kernels (and >>> possibly elsewhere). None of the alternatives is >>> automatic. None are independent of software choices >>> that must be made by each operating system. >> >> >> Yes. They all require somebody to be interested in doing the work. >> >> Warner >> >> >> >