Date: Mon, 12 Jun 2017 19:01:44 -0700 From: Mark Millard <markmi@dsl-only.net> To: Russell Haley <russ.haley@gmail.com> Cc: "freebsd-arm@freebsd.org" <freebsd-arm@freebsd.org> Subject: Re: Creating armv7 MACHINE_ARCH Message-ID: <C5D08468-DB46-4972-BFE2-44559A5EAB88@dsl-only.net> In-Reply-To: <CABx9NuRJLgc0cSsykrSbJi=fVq-jfOi7ZEiZ5LRHG5tqibZ_2Q@mail.gmail.com> References: <CANCZdfpUjPBRpxpmjtwK-wpiK=%2BwHscS4UmVeatrE7vrm260tw@mail.gmail.com> <20170612152808.6094931.74364.27128@gmail.com> <CANCZdfrxTo8vLsnjU_VerO%2B3%2BU=06cok7%2BuKba3FM8_nXFozhQ@mail.gmail.com> <B19EDB95-2A23-4F8F-8414-3F4E0E65AC4B@dsl-only.net> <CABx9NuQTOkf6HK=RacUCBR=W_WDfgZwbVHYwsRdx0YJd=zr51w@mail.gmail.com> <2A90A527-7DCA-4442-9322-0EA96236583C@dsl-only.net> <3CC8DE8A-CCF2-4856-A43E-6B259BDE8B2C@dsl-only.net> <CANCZdfp6cnXdHxSQGTXHq4Md4Jh6=u4Af_rDM3k_RgoN%2BFrXWA@mail.gmail.com> <CABx9NuRJLgc0cSsykrSbJi=fVq-jfOi7ZEiZ5LRHG5tqibZ_2Q@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2017-Jun-12, at 4:10 PM, Russell Haley <russ.haley at gmail.com> = wrote: > Okay, feel free to ignore me, I'm not going to get the time drill into > the source code for my own questions so I don't expect anyone else > too. However, I'll ask anyway. I'm too confused to try and inline > these questions. Lets see if I understand: >=20 > - armv7 does not support 64 bit instructions (according to Wikipedia? > I claim no expertise.) It does not support AArch64 instructions but does support AArch32 instructions (or is a good approximation). AArch32 was actually created later to be armv7-a like. > - FreeBSD has an armv6 "architecture" that is supports armv6 and armv7 > on Pre-Cortex-A-53 processors that is not supported on A-53 through > the emulated AArch32. There are actually architectures (by ARM's definitions). . . ARMv6, ARMv6T2, ARMv6Z, ARMv6K, ARMV6-M ARMv7-M, ARMv7E-M, ARMv7-R, ARMv7-A ARMv8-A, ARMv8.1-A ARMv8.2-Am ARMV8.3-A, ARMv8-R, ARMv8-M I'm going to largely ignore much of that variation in structure. ARM has continued to produce/make new variations for both 32-bit and 64-bit in overlapping time frames. So the "Pre-Cortex-A53" presumes to much about relative timing. Technically the old RPI2s (V1.1 and before) have Cortex-A7's and the new V1.2's have Cortex-A53's. RPI3 is also Cortex-A53 based. Cortex-A7 includes an implementation of armv7-A architecture (but is not the only one). Cortex-A53 includes an implementation of armv8-A architecture=20 (but is not the only one). (Both include other things as well. And the System On A Chip has even more stuff than the Cortex-A<?>'s in question.) Cortex-A<?>'s are specific CPUs/cores that are also examples of the specific architecture(s) they implement. If a kernel supports armv7-a it can support a variety of CPUs that all implement armv7-a architecture. It may not support things in the CPUs that are beyond that architecture. If a kernel supports armv8-a it can support a variety of CPUs that all implement armv8-a architecture. It may not support things in the CPUs that are beyond that architecture. Note there are also issues of support of the System On a Chip involved as well. More than ARM is involved overall. > - Cortex A-53 can support armv8 (AArch64) and armv7 (AArch32) = instructions As I remember armv8-a specifies: A) Optional support: AArch64 (Cortex-A32 is ARMv8-A archtecture but only 32-bit) (ARMv8-R architecture is always only 32-bit as I understand) (ARMv8-M architecture is always only 32-bit as I understand) B) Optional support: AArch32 (Cortex-A35/53/57/72/73 are ARMv8-A and have both) (Cortex-A55/75 are ARMV8.2-A architecture and have both) (But variants can be produced that omit AArch32.) C) Various things specific to armv8-a might go beyond what AArch64 and AArch32 specify. Note: AArch32 has 2 instruction sets (A32/ARM and T32/Thumb), like armv7-a does. AArch64 has A64. (I'm not giving thumb version numbers here but there are versioned vintages.) So 1 architecture (armv8-a) has at least 3 instruction sets when both AArch64 and AArch32 are implemented. AArch64 state: A64 AArch32 state: A32 and T32 There can be context switching between these states as I understand. (I'll not get into the NEON distinctions and such so the above is simplified.) armv8-a does not necessarily uniquely identify the interrupt controller or its software interface, as an example of variation that is not an instruction set issue: register interfaces to other hardware. ARM provides GIC-400 and GIC-500 and others. But the controller does not have to come from ARM. While Cortex-A53 has a default/reference interrupt controller(s), it is possible to build variations that use other ones as I understand. Similar points go for Cortex-A7. The other parts of a System On a Chip are for non-arm aspects and may have some mixed degree on commonality independently of ARM. But the kernel may deal with such specifics as well. > - The current proposal is to split the armv6 and armv7 into their own > "architectures" ARM defines these as distinct architectures. That is not a FreeBSD specific terminology. But much of the armv6 is common with armv7. armv7 architecture has more than armv6 architecture does: it is an update. Having armv7 implemented separately in FreeBSD means more completely/correctly identifying what is there compared to pretending it is an armv6. > FreeBSD has an Arm 64 bit kernel build. I don't see what the > TARGET_ARCH settings in the wiki and know little about it, but will > conjecture that it doesn't have a TARGET_ARCH=3Darmv8 (please correct = me > if I'm wrong). FreeBSD has TARGET=3Darm64 TARGET_ARCH=3Daarch64 . That combination is for armv8-A with AArch64 currently. Cortex-A32 is not supported because it lacks AArch64. FreeBSD's TARGET=3Darm64 TARGET_ARCH=3Daarch64 may grow to span armv8.1 or such as well for all I know. Cortex-A53 is one example of an armv8 implementation that the kernel supports (presuming certain interrupt controllers and such). There are others. > What I was trying to ask was: is the kernel development moving in a > direction that clearly indicates the differences in the instructions > vs the architectures (or have I grossly simplified the problem)? Will > it be possible to target a Cortex-A53 and build for 32 or 64 bit > support? Or is this just to fix RPi? Instruction sets are part of an architecture but not all of it. They are not independent of architecture. Talking of an overall architecture includes covering the instruction set(s). One architecture can have multiple instruction sets as part of it. TARGET=3Darm64 TARGET_ARCH=3Daarch64 may get a "lib32" (compat32) implementation someday if someone is motivated to do the work. That would enable AArch32 user processes. What exists now and will in the future is mostly tied to choices of volunteers that say "I'm going to provide <whatever>" and then manage to complete whatever it was. (Clearly a "we are going to" is also an option.) A similar points would go for TARGET_ARCH=3Darmv7 supporting something like Cortex-A53 in AArch32 mode (Raspian like). So at this point no one knows if or when as far as I know. armv6 and armv7 architectures are strongly overlapping for what is in armv6. The GENERIC kernel for what is currently a TARGET_ARCH=3Darmv6 build actually builds for armv7 and will not work on armv6. One has to use a different kernel configuration than GENERIC for armv6 (such as the one for the older RPI vintage that was armv6 based). Identifying an armv7-a build as a armv6 build has odd consequences for software that tries to figure out what can be depended on. > In terms of Raspbian, I had assumed they were just supporting Aarch32 > across both processor models. Many of the drivers would be the same > source if they share components so I would think it would be "simple". > I didn't see anything in my brief look at it today to indicate > otherwise. For Cortex-A53: Raspian supports just AArch32 state, not AArch64 state for rpi2 V1.2 and RPI3. (I do not know if early boot is temporarily AArch64 or not.) For Cortex-A7: Cortex-A7 predates AArch32 and is armv7-a directly but AArch32 was designed to appear to be be nearly the same as armv7-a. So Raspian is supporting armv7-a directly for rpi2 V1.1 and V1.0. =3D=3D=3D Mark Millard markmi at dsl-only.net On Mon, Jun 12, 2017 at 2:07 PM, Warner Losh <imp@bsdimp.com> wrote: >=20 >=20 > On Mon, Jun 12, 2017 at 2:35 PM, Mark Millard <markmi@dsl-only.net> = wrote: >>=20 >>=20 >> On 2017-Jun-12, at 1:00 PM, Mark Millard <markmi at dsl-only.net> = wrote: >>=20 >>> On Mon, Jun 12, 2017 at 1:00 PM, Mark Millard <markmi at = dsl-only.net> >>> wrote: >>>> On 2017-Jun-12, at 12:16 PM, Russell Haley <russ.haley at = gmail.com> >>>> wrote: >>>>=20 >>>>> On Mon, Jun 12, 2017 at 10:36 AM, Mark Millard <markmi at >>>>> dsl-only.net> wrote: >>>>>>=20 >>>>>> On 2017-Jun-12, at 8:39 AM, Warner Losh <imp at bsdimp.com> = wrote: >>>>>>=20 >>>>>>> . . . >>>>>>>=20 >>>>>>> Plus, we aren't quite doing what Ian wanted. He wanted a full >>>>>>> rename. The >>>>>>> proposal on the able is to add an armv7 TARGET_ARCH in 12. Not = to >>>>>>> rename or >>>>>>> remove armv6. Sadly, that will still be there since the RPI >>>>>>> foundation >>>>>>> keeps finding new ways to repackage the rpi into new boards that = are >>>>>>> just >>>>>>> too cheap to ignore. >>>>>>=20 >>>>>> On 2017-Jun-12, at 6:59 AM, Andrew Turner <andrew at = fubar.geek.nz> >>>>>> wrote: >>>>>>=20 >>>>>>> I like this. My understanding is adding armv7 would also fix = many of >>>>>>> the currently broken ports that assume they are being built for = armv7 as >>>>>>> many Linux distros target ARMv7+. >>>>>>>=20 >>>>>>> It should also be noted the GENERIC kernel is likely to only = ever >>>>>>> target ARMv7+ even without an armv7 TARGET_ARCH. >>>>>>=20 >>>>>>=20 >>>>>> Hopefully the choices related to TARGET and TARGET_ARCH >>>>>> for armv7 end up identifying the context to port builds >>>>>> so that many would just automatically do the right thing. >>>>>>=20 >>>>>>=20 >>>>>> As for GENERIC: >>>>>>=20 >>>>>> powerpc has. . . >>>>>>=20 >>>>>> TARGET=3Dpowerpc TARGET_ARCH=3Dpowerpc and GENERIC >>>>>> TARGET=3Dpowerpc TARGET_ARCH=3Dpowerpc64 and GENERIC64 >>>>>>=20 >>>>>> So there is precedent for more than one GENERIC* >>>>>> for a family, with which one being appropriate >>>>>> being based on TARGET_ARCH. >>>>>>=20 >>>>>> For powerpc TARGET=3Dpowerpc implicitly uses >>>>>> TARGET_ARCH=3Dpowerpc when TARGET_ARCH is not >>>>>> specified (if I remember right). Which should >>>>>> be the default for armv6 vs. armv7 might go >>>>>> the other direction (TARGET_ARCH=3Darmv7 by >>>>>> default). >>>>>>=20 >>>>>>=20 >>>>>> Side note: >>>>>>=20 >>>>>> A caution about talking about "rpi2" as >>>>>> an example. . . >>>>>>=20 >>>>>> Raspberry Pi 2 Model B V1.2 is Cortex-A53 based >>>>>> (so arm64/aarch64). (A BCM2837, not a BCM2836.) >>>>>> This dates about to something like 2014 based >>>>>> on the pictures showing the (c) notice on the >>>>>> boards. >>>>>>=20 >>>>>> V1.1 and before were armv7 (BCM2836) based. >>>>>>=20 >>>>>> Unless a kernel and world are made that can >>>>>> also configure/handle a Cortex-A53 in a >>>>>> armv7-like manor there will be two different >>>>>> GENERIC builds in order to span the "rpi2" >>>>>> family, based on just V1.2+ vs. V1.1 and >>>>>> before. >>>>>>=20 >>>>>> (A single, modern distribution of the official >>>>>> Raspbian software for the rpi2 does support >>>>>> all the V1.x boards if I understand right.) >>>>>=20 >>>>> I am confused. I don't see any documentation about Raspbian = supporting >>>>> 64 bit? >>>>=20 >>>> 64-bit Cortex-A53's can be configure to operate in a >>>> 32-bit mode (AArch32). Raspian does that for RPI2 V1.2 >>>> and for RPI3. >>>>=20 >>>> Raspian does not (yet?) support a 64-bit mode (AArch64). >>>>=20 >>>> The Cortex-A53 can support either. As I understand it >>>> is possible for an OS to allow a user processes to be >>>> one or the other, different processes using the different >>>> modes. That does not mean that all operating systems >>>> bother to. >>>>=20 >>>> If I remember right the official Ubuntu for an ODroid-C2 >>>> allows both AArch64 and AArch32 user programs (and >>>> so processes, with shared library types tracking). >>>>=20 >>>>> =46rom Arm at >>>>> = https://www.arm.com/products/processors/cortex-a/cortex-a53-processor.php:= >>>>> "The Cortex-A53 supports the full ARMv8-A architecture. It not = only >>>>> runs 64-bit applications also seamlessly and efficiently runs = legacy >>>>> ARM 32-bit applications." >>>>>=20 >>>>> I assume that means it handles armv7-A without issue? (In fact, = many >>>>> on this board know it does) >>>>=20 >>>> I've not gone through the details but targeting AArch32 >>>> probably means targeting a subset of armv7. Or may be >>>> to support both requires targeting a common subset of both. >>>> (My guess is that AArch32 is the definition of a common >>>> subset for 32-bit, at least for user processes.) >>>>=20 >>>> Raspian targets just AArch32 on armv7 and Cortex-A53 >>>> (user space). (If I've got the definition of AArch32 >>>> right: otherwise a common subset.) >>>>=20 >>>> FreeBSD targets armv7 and AArch64 separately (via >>>> separate GENERIC kernels). I'm not aware of FreeBSD >>>> targeting AArch32 at all on cores capable of AArch64. >>>> FreeBSD possibly does not restrict itself to AArch32 >>>> (user processes) on armv7 if AArch32 is really a >>>> subset. >>>=20 >>> I thought all 64 bit Arm instructions are defined in armv8? >>=20 >> (I assume you are not trying to indicate armv8.1, armv8.2 >> and such. Cortex-A53 is older than those and so does not >> have the newer things involved.) >>=20 >> That Cortex-A53 allows armv8 64-bit arm code is not in >> dispute. But the operating system in involved in setting >> up what will actually work based on how it configures >> things and operates. Much of this is the kernel. >=20 >=20 > Correct. >=20 >>=20 >> Cortex-A53 also supports AArch32, i.e., 32 bit instructions. >> So that the 64-bit instructions all being there does not >> of itself prevent using a 32-bit mode instead. >>=20 >> (The kernel likely has to deal with more specifics of >> processor variations than user code does not. My notes >> are really about the user process model, not the all >> the kernel details.) >>=20 >> Raspian deals with armv7's that have no 64-bit support >> and with Cortex-A53's that do. It presents a user-process >> model that is 32-bit only, even on Cortex-A53's that allow >> for 64-bit (but do not required user programs to be AArch64 >> code). >>=20 >> Ubuntu for ODroid-C2 does not deal with armv7's but does >> allow both 64-bit (AArch64) and 32-bit (AArch32) user >> processes as I remember, on its Cortex-A53's. >>=20 >> FreeBSD armv7 does not support Cortext-A53 or anything >> that allows 64-bit (that allows AArch64). This is a kernel >> level issue. >=20 >=20 > Not a hugely difficult issue to fix, but one nobody had fixed... >=20 >>=20 >> FreeBSD aarch64 does not support having AArch32 user >> processes. Nor does its kernel support processors that >> do not support aarch64 (so it does not support armv7). >=20 >=20 > Executing a 32-bit /bin/cat on aarch64 level support exists outside = the > tree, according to the hallway track at BSDcan, so it will only be a = matter > of time before compat32 exists there I think. >=20 > There's a further complication is that the aarch32 unit of aarch64 > processors is optional. Not all of them have it, so that can be a = problem... > IIRC, the early aarch64 targets didn't have this feature... >=20 >>=20 >> These are simply examples of different choices about >> what combinations of the technical possibilities to >> put effort into supporting in the kernels (and >> possibly elsewhere). None of the alternatives is >> automatic. None are independent of software choices >> that must be made by each operating system. >=20 >=20 > Yes. They all require somebody to be interested in doing the work. >=20 > Warner >=20 >=20 >=20
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?C5D08468-DB46-4972-BFE2-44559A5EAB88>