Date: Sat, 2 Mar 2013 10:10:44 -0800 From: Tim Kientzle <tim@kientzle.com> To: Ian Lepore <ian@freebsd.org> Cc: freebsd-arm@freebsd.org Subject: Re: PHYSADDR Message-ID: <AC642AAD-8D8C-429E-AE3B-A34241DA2517@kientzle.com> In-Reply-To: <1362246634.1195.178.camel@revolution.hippie.lan> References: <E886046B-1612-425B-902B-72D4B0E93618@freebsd.org> <1362068453.1195.40.camel@revolution.hippie.lan> <674A08B3-6600-4B77-8511-9EF54E4B9B1F@FreeBSD.org> <8FEA3237-8ABF-4564-B672-4B4C0C6EF291@kientzle.com> <1362155632.1195.120.camel@revolution.hippie.lan> <5B622D1B-4EAE-4184-A194-DD14083A48B6@kientzle.com> <1362246634.1195.178.camel@revolution.hippie.lan>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mar 2, 2013, at 9:50 AM, Ian Lepore wrote: > On Fri, 2013-03-01 at 09:19 -0800, Tim Kientzle wrote: >> On Mar 1, 2013, at 8:33 AM, Ian Lepore wrote: >>=20 >>> On Thu, 2013-02-28 at 09:44 -0800, Tim Kientzle wrote: >>>> On Feb 28, 2013, at 8:58 AM, Tim Kientzle wrote: >>>>=20 >>>>>=20 >>>>> On Feb 28, 2013, at 8:20 AM, Ian Lepore wrote: >>>>>=20 >>>>>> On Wed, 2013-02-27 at 22:27 -0800, Tim Kientzle wrote: >>>>>>> Starting to look at what is needed for a Generic ARM kernel. >>>>>>> There's a lot here; I sincerely hope I'm not the only one=85 ;-) >>>>>>>=20 >>>>>>> First up: Can we get rid of PHYSADDR? >>>>>>>=20 >>>>>>=20 >>>>>> If you mean, can we get rid of it within the runtime kernel, I'd = say >>>>>> yes, because we can use a global variable instead which is easily >>>>>> settable in the entry code. >>>>>=20 >>>>> It doesn't seem to be used in the runtime kernel. As far as >>>>> I can see, it's purely a bootstrap concept. >>>>>=20 >>>=20 >>> Well, it's used to set up the early-init page tables in locore.s = then >>> again to set up the real page tables and related things in initarm() = and >>> then I think it isn't used after that, so I should have said "within = the >>> kernel init". The main point I was getting at is that I don't think = we >>> need a compile-time constant of any sort related to the physical = address >>> at which the kernel is loaded. We can get the physical address of = the >>> entry point (_start) using pc-relative math, and we can get the = linker >>> to give us a constant symbol which is the offset of the _start = symbol >>> within the load image. So we can get the load address at runtime >>> without guessing what low-order bits to mask. >>=20 >> Good. That's the approach I've been eyeing as well; I'm >> glad you agree that makes sense. >>=20 >>>=20 >>>>>> On the other hand, I've been working >>>>>> towards getting that value set correctly in the kernel elf = headers at >>>>>> link time. >>>>=20 >>>> A-ha! I think I just figured something out. >>>>=20 >>>> How would the following work: >>>>=20 >>>> * Rename PHYSADDR to KERNPHYSADDR_BASE >>>>=20 >>>> * in the top of locore.s, we have a single conditional: >>>>=20 >>>> #ifdef KERNPHYSADDR_BASE >>>> _kpa_base =3D KERNPHYSADDR_BASE; >>>> #else >>>> _kpa_base =3D pc & 0xF0000000; >>>> #endif >>>>=20 >>>> I think this would DTRT on all of the configurations >>>> we currently have in SVN. >>>=20 >>> Hmm, so the basic assumption is that every SoC will have some = physical >>> memory aligned to a 256mb boundary. >>=20 >> I'm assuming this only for ARM systems supported by the GENERIC >> kernel. >>=20 >> Ss far as I can tell, the 256mb boundary assumption works for >> everything currently in SVN. But the code above does allow you >> to custom-build a kernel for a system that doesn't satisfy that; >> you just won't be able to run the GENERIC kernel there. >>=20 >> Eventually, it seems we might pull this information out of the >> FDT, but I'm not yet ready to tackle FDT parsing in locore.S. ;-) >>=20 >> Of course, I'm not certain that it will matter when we're done. >> If we only need PHYSADDR to set up the MMU paging, >> then we just need to round the _start address down to >> the next page boundary, don't we? >>=20 >>> E.G., there'll never be a SoC with >>> memory at 0xN1000000 that doesn't have memory at 0xN0000000. I'm = not >>> sure that's a safe assumption given things like the rpi where the = gpu >>> carves off some memory for itself and gives the rest to the arm. It >>> works with the way rpi carves up the ram, but I could see similar >>> designs that wouldn't work. >>>=20 >>>>=20 >>>> * We redefine KERNPHYSADDR to be an *offset* >>>> against _kpa_base. Then we could negotiate a single >>>> offset (64k?) that works well on many platforms and use >>>> that for the GENERIC kernel. Boot loaders would be >>>> responsible for loading the kernel at an address that >>>> preserves the KPA_OFFSET. The KPA_OFFSET would >>>> be used in the ELF headers. >>>>=20 >>>> Then there are routine code transformations to use _kpa_base >>>> instead of the compile-time symbol and to use >>>> _kpa_base + KERNPHYSADDR instead of KERNPHYSADDR. >>>>=20 >>>> Does this sound reasonable as a starting point? >>>>=20 >>>=20 >>> There are even more assumptions here about what would work in every >>> case. Given the basic sequence of boot2->u-boot->ubldr->kernel, = every >>> one of those components has to load the next component at the = physical >>> address it's linked for, and that has to not overlap the addresses = being >>> used by the thing doing the loading. The MMU is off during all of = this >>> so we can't just map our way out of any conflicts. >>=20 >> I've given up entirely on the first two of these being generic. >>=20 >> I think we have a shot at making the kernel itself generic, >> and maybe ubldr could be made truly PIC, but the earlier >> boot stages are always going to be highly board-specific. >>=20 >> So conflicts between the various pieces aren't really >> my primary worry at the moment. Since we'll have to >> customize the early boot pieces anyway, we can resolve >> those on a case-by-case basis. >>=20 >> The ELF load address vs. where physical memory is located >> seems the sticky point. Any ideas for getting around that? >> I feel like I have enough ideas to start chipping away at >> locore.S if I could just figure out a strategy for the ELF >> load address issue. >>=20 >> (Of course, I still don't understand why the test image I've >> been playing with seems able to load the same ubldr on >> both RPi and BBone and that ubldr seems to have no trouble >> loading a kernel into memory.) I finally figured this out. RPi echoes memory (so 0x80000000-0xA0000000 is another window on 0x00000000-0x20000000), so my BeagleBone boot bits linked for the 0x80000000-0x90000000 range actually did work on RPi. (Which is annoying; I was hoping the different memory ranges on these two boards would help me to catch memory layout problems.) > We may not have control over anything before ubldr. Not everything is > an eval board that you can easily build a custom u-boot for. =20 Yep. Full agreement here. > I'm not sure its safe to assume that (entry-pc & 0xfffff000) is the > beginning of the kernel; it's true now but need not be so. But that's > no big deal, we can tweak the linker script to give us the offset of = the > _start symbol so it'll work no matter what. Patches? ;-) > The more I ponder the fixed offset concept for a generic kernel the = more > it seems that it might be viable, providing that we require the use of > ubldr. Then we can make sure that ubldr is always linked at an offset > that doesn't overlap the kernel load offset. An offset number like = 1mb > or 4mb might work well for the kernel; it leaves lots of space for = ubldr > to be loaded down low in memory. I think putting the loader at a = lower > address than the kernel is required, because there's no upper bound on > the kernel size (I did a 27MB kernel last month -- it contains a huge > embedded md filesystem image). Thanks for pointing this out. I've been consistently putting ubldr in high memory but hadn't realized kernels varied that much in size. I'll rethink that. > This just pushes the real problem into ubldr, though, and that becomes > the non-generic component that has to be linked at a different = physical > address for each SoC. A kernel could still be loaded without ubldr, = but > it may need to be built in a non-generic way for some platforms. Among the many things I'd like to try: see if there's a way to build ubldr as a PIC raw binary (instead of an ELF image). That might help in a lot of case. Tim
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?AC642AAD-8D8C-429E-AE3B-A34241DA2517>