From owner-freebsd-ppc@FreeBSD.ORG Tue Sep 9 16:11:10 2014 Return-Path: Delivered-To: freebsd-ppc@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D8EEDAD for ; Tue, 9 Sep 2014 16:11:10 +0000 (UTC) Received: from mail.xcllnt.net (mail.xcllnt.net [50.0.150.214]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9BF2116A for ; Tue, 9 Sep 2014 16:11:10 +0000 (UTC) Received: from [172.29.13.157] ([66.129.239.11]) (authenticated bits=0) by mail.xcllnt.net (8.14.9/8.14.9) with ESMTP id s89GB8ij011045 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO) for ; Tue, 9 Sep 2014 09:11:10 -0700 (PDT) (envelope-from marcel@xcllnt.net) From: Marcel Moolenaar Content-Type: multipart/signed; boundary="Apple-Mail=_CDDAE297-F448-44F9-97F2-68B0A76CED4F"; protocol="application/pgp-signature"; micalg=pgp-sha1 Subject: [booke] preloading limited to initial mapping in kernel Message-Id: Date: Tue, 9 Sep 2014 09:11:02 -0700 To: freebsd-ppc@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) X-Mailer: Apple Mail (2.1878.6) X-BeenThere: freebsd-ppc@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Porting FreeBSD to the PowerPC List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Sep 2014 16:11:10 -0000 --Apple-Mail=_CDDAE297-F448-44F9-97F2-68B0A76CED4F Content-Type: multipart/mixed; boundary="Apple-Mail=_9CBBB915-A277-43BE-8F77-3E9533C47C96" --Apple-Mail=_9CBBB915-A277-43BE-8F77-3E9533C47C96 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii All, The kernel creates a single 16MB mapping in locore.S under the assumption that what got preloaded fits that size. With the metadata at the end of what's preloaded, the kernel will simply fail to boot is the total amount preloaded is larger than the initial mapping. Juniper fixed this problem by having the loader create the initial mapping based on the total size preloaded and communicate this to the kernel by virtue of the number of TLBs used. A change was made in locore.S to detect Juniper's loader and skip over the entire TLB1 fiddling. Juniper is currently porting Junos onto the latest FreeBSD version using mostly stock FreeBSD bits and problem described above is back again. Juniper has 2 options: 1. Make the same change to FreeBSD's loader as it did before. The problem with this approach is that newer U-Boot versions use the first few TLB1 entries for itself, meaning that the loader has to jump through a few hoops to make sure it can use the first N entries without destroying the entries used by U-Boot and itself. 2. Change locore.S and map enough to cover all the preloaded bits. The problem here is that the kernel does not know how much is preloaded. While it's easy enough to tell the kernel that, the problem is that the kernel must have a reliable way to detect by which loader it was loaded so that it knows whether the size of preloading is in whatever register we dedicate for that. Juniper's loader swizzled the registers so that the kernel can check for this. To be precise: FreeBSD's loader passes the metadata pointer (MDP) in r3. Juniper's loader sets r3 to 0 and passes the MDP in r4. Note also that I added support for booting directly from U-Boot. I'd like that to continue to work for as much as is possible. I'd like suggestions as to what approach to use. I played with the first (change the loader) and it can be made to work. It just doesn't feel right. I attached the patch for reference. Thanks, -- Marcel Moolenaar marcel@xcllnt.net --Apple-Mail=_9CBBB915-A277-43BE-8F77-3E9533C47C96 Content-Disposition: attachment; filename=booke.diff Content-Type: application/octet-stream; name="booke.diff" Content-Transfer-Encoding: 7bit Index: sys/boot/powerpc/uboot/tlb.c =================================================================== --- sys/boot/powerpc/uboot/tlb.c (revision 0) +++ sys/boot/powerpc/uboot/tlb.c (revision 0) @@ -0,0 +1,215 @@ +/*- + * Copyright (C) 2007-2014 Juniper Networks, Inc. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include +__FBSDID("$FreeBSD$"); + +#include +#include + +#define BOOKE +#define BOOKE_E500 +#include +#include + +#include "libuboot.h" + +#define MAX_ENTRIES 4 + +static __inline uint32_t +ilog2(uint32_t num) +{ + uint32_t lz; + + __asm __volatile("cntlzw %0, %1" : "=r" (lz) : "r" (num)); + return (31 - lz); +} + +static void +tlb1_write_entry(int idx, uint32_t pa, uint32_t va, uint32_t sz) +{ + uint32_t mas0, mas1, mas2, mas3, pvr; + + pvr = mfspr(SPR_PVR) >> 16; + sz = ilog2(sz) / 2 - 5; + + mas0 = MAS0_TLBSEL(1) | MAS0_ESEL(idx); + mas1 = MAS1_VALID | MAS1_IPROT | + ((sz << MAS1_TSIZE_SHIFT) & MAS1_TSIZE_MASK); + mas2 = (va & MAS2_EPN_MASK) | MAS2_M; + mas3 = (pa & MAS3_RPN) | MAS3_SR | MAS3_SW | MAS3_SX; + + mtspr(SPR_MAS0, mas0); + __asm volatile("isync"); + mtspr(SPR_MAS1, mas1); + __asm volatile("isync"); + mtspr(SPR_MAS2, mas2); + __asm volatile("isync"); + mtspr(SPR_MAS3, mas3); + __asm volatile("isync"); + mtspr(SPR_MAS7, 0); + switch (pvr) { + case FSL_E500mc: + case FSL_E5500: + __asm volatile("isync"); + mtspr(SPR_MAS8, 0); + break; + default: + break; + } + __asm volatile("isync"); + __asm volatile("tlbwe"); + __asm volatile("isync"); + __asm volatile("msync"); +} + +static int +find_curr_tlb1_entry(void) +{ + uint32_t addr; + uint32_t pid, mas0, idx; + + addr = (uint32_t) find_curr_tlb1_entry; + pid = mfspr(SPR_PID0); + pid = pid << MAS6_SPID0_SHIFT; + mtspr(SPR_MAS6, pid); + __asm volatile("isync"); + __asm volatile("tlbsx 0,%0" :: "r"(addr)); + mas0 = mfspr(SPR_MAS0); + mas0 = mas0 >> 16; + idx = mas0 & 0x3f; + return (idx); +} + +static void +tlb1_invalidate(int skip_idx) +{ + uint32_t mas0, mas1, mas2, mas3, pvr; + int idx, tlb1_size; + + pvr = mfspr(SPR_PVR) >> 16; + tlb1_size = mfspr(SPR_TLB1CFG) & TLBCFG_NENTRY_MASK; + + printf("XXX: %s: skip_idx=%d, tlb1_size=%d\n", __func__, + skip_idx, tlb1_size); + + for (idx = 0; idx < tlb1_size; idx++) { + mas0 = MAS0_TLBSEL(1) | MAS0_ESEL(idx); + mtspr(SPR_MAS0, mas0); + + __asm __volatile("isync; tlbre"); + + mas1 = mfspr(SPR_MAS1); + mas2 = mfspr(SPR_MAS2); + mas3 = mfspr(SPR_MAS3); + + printf("XXX: %02d: mas1=%04x mas2=%04x mas3=%04x\n", idx, + mas1, mas2, mas3); + } + + for (idx = 0; idx < tlb1_size; idx++) { + if (idx == skip_idx) + continue; + + printf("XXX: %s: idx=%d\n", __func__, idx); + + mas0 = MAS0_TLBSEL(1) | MAS0_ESEL(idx); + mas1 = 0x00; + mas2 = 0x00; + mas3 = 0x00; + + mtspr(SPR_MAS0, mas0); + __asm volatile("isync"); + mtspr(SPR_MAS1, mas1); + __asm volatile("isync"); + mtspr(SPR_MAS2, mas2); + __asm volatile("isync"); + mtspr(SPR_MAS3, mas3); + __asm volatile("isync"); + mtspr(SPR_MAS7, 0); + switch (pvr) { + case FSL_E500mc: + case FSL_E5500: + __asm volatile("isync"); + mtspr(SPR_MAS8, 0); + break; + default: + break; + } + __asm volatile("isync"); + __asm volatile("tlbwe"); + __asm volatile("isync"); + __asm volatile("msync"); + } +} + +vm_size_t +uboot_map(vm_paddr_t pa, vm_offset_t va, vm_size_t sz) +{ + vm_size_t pgs[MAX_ENTRIES]; + vm_size_t mapped; + vm_size_t pgsz; + int idx, maxidx; + + sz = roundup(sz, 1048576); + + mapped = 0; + idx = 0; + pgsz = 64*1024*1024; + while (mapped < sz) { + while (mapped < sz && idx < MAX_ENTRIES) { + while (pgsz > (sz - mapped)) + pgsz >>= 2; + pgs[idx++] = pgsz; + mapped += pgsz; + } + /* We under-map. Correct for this. */ + if (mapped < sz) { + while (idx > 0 && pgs[idx - 1] == pgsz) { + idx--; + mapped -= pgsz; + } + /* XXX We may increase beyond out starting point. */ + pgsz <<= 2; + pgs[idx++] = pgsz; + mapped += pgsz; + } + } + + tlb1_invalidate(find_curr_tlb1_entry()); + + maxidx = idx; + for (idx = 0; idx < maxidx; idx++) { + pgsz = pgs[idx]; + tlb1_write_entry(idx, pa, va, pgsz); + pa += pgsz; + va += pgsz; + } + + printf("> mapped=%#x\n", mapped); + + return (mapped); +} Index: sys/boot/powerpc/uboot/Makefile =================================================================== --- sys/boot/powerpc/uboot/Makefile (revision 285901) +++ sys/boot/powerpc/uboot/Makefile (working copy) @@ -10,7 +10,7 @@ MAN= # Architecture-specific loader code SRCS= start.S conf.c vers.c -SRCS+= ucmpdi2.c +SRCS+= tlb.c ucmpdi2.c CFLAGS+= -DBOOTPROG=\"${PROG}\" Index: sys/boot/arm/uboot/conf.c =================================================================== --- sys/boot/arm/uboot/conf.c (revision 285901) +++ sys/boot/arm/uboot/conf.c (working copy) @@ -92,3 +92,9 @@ struct console *consoles[] = { &uboot_console, NULL }; + +vm_size_t +uboot_map(vm_paddr_t pa __unused, vm_offset_t va __unused, vm_size_t sz) +{ + return (sz); +} Index: sys/boot/uboot/lib/libuboot.h =================================================================== --- sys/boot/uboot/lib/libuboot.h (revision 285901) +++ sys/boot/uboot/lib/libuboot.h (working copy) @@ -62,6 +62,7 @@ ssize_t uboot_copyin(const void *src, vm_offset_t ssize_t uboot_copyout(const vm_offset_t src, void *dest, const size_t len); ssize_t uboot_readin(const int fd, vm_offset_t dest, const size_t len); extern int uboot_autoload(void); +vm_size_t uboot_map(vm_paddr_t pa, vm_offset_t va, vm_size_t size); struct preloaded_file; struct file_format; Index: sys/boot/uboot/common/metadata.c =================================================================== --- sys/boot/uboot/common/metadata.c (revision 285901) +++ sys/boot/uboot/common/metadata.c (working copy) @@ -41,10 +41,7 @@ __FBSDID("$FreeBSD$"); #include "api_public.h" #include "bootstrap.h" #include "glue.h" - -#if defined(LOADER_FDT_SUPPORT) #include "libuboot.h" -#endif static int md_getboothowto(char *kargs) @@ -251,11 +248,11 @@ md_load(char *args, vm_offset_t *modulep) struct preloaded_file *kfp, *bfp; struct preloaded_file *xp; struct file_metadata *md; - struct bootinfo *bip; - vm_offset_t kernend; - vm_offset_t addr; - vm_offset_t envp; - vm_offset_t size; + vm_paddr_t kernbase; + vm_paddr_t kernend; + vm_paddr_t addr; + vm_paddr_t envp; + vm_size_t size; vm_offset_t vaddr; #if defined(LOADER_FDT_SUPPORT) vm_offset_t dtbp; @@ -290,33 +287,30 @@ md_load(char *args, vm_offset_t *modulep) /* Try reading the /etc/fstab file to select the root device */ getrootmount(rootdevname); - /* Find the last module in the chain */ - addr = 0; + /* Find the lowest and highest addresses used */ + kernend = 0; + kernbase = ~0; for (xp = file_findfile(NULL, NULL); xp != NULL; xp = xp->f_next) { - if (addr < (xp->f_addr + xp->f_size)) - addr = xp->f_addr + xp->f_size; + if (kernend < (xp->f_addr + xp->f_size)) + kernend = xp->f_addr + xp->f_size; + if (xp->f_addr < kernbase) + kernbase = xp->f_addr; } - /* Pad to a page boundary */ - addr = roundup(addr, PAGE_SIZE); + kernend = roundup(kernend, PAGE_SIZE); /* Copy our environment */ - envp = addr; - addr = md_copyenv(addr); + envp = kernend; + kernend = md_copyenv(kernend); + kernend = roundup(kernend, PAGE_SIZE); - /* Pad to a page boundary */ - addr = roundup(addr, PAGE_SIZE); - #if defined(LOADER_FDT_SUPPORT) /* Handle device tree blob */ - dtbp = addr; - dtb_size = fdt_copy(addr); - - /* Pad to a page boundary */ + dtbp = kernend; + dtb_size = fdt_copy(kernend); if (dtb_size) - addr += roundup(dtb_size, PAGE_SIZE); + kernend += roundup(dtb_size, PAGE_SIZE); #endif - kernend = 0; kfp = file_findfile(NULL, "elf32 kernel"); if (kfp == NULL) kfp = file_findfile(NULL, "elf kernel"); @@ -336,10 +330,15 @@ md_load(char *args, vm_offset_t *modulep) file_addmetadata(kfp, MODINFOMD_KERNEND, sizeof kernend, &kernend); /* Figure out the size and location of the metadata */ - *modulep = addr; + *modulep = addr = kernend; size = md_copymodules(0); - kernend = roundup(addr + size, PAGE_SIZE); + kernend += roundup(size, PAGE_SIZE); + vaddr = kernbase - __elfN(relocation_offset); + size = kernend - kernbase; + size = uboot_map(kernbase, vaddr, size); + kernend = kernbase + size; + /* Provide MODINFOMD_KERNEND */ md = file_findmetadata(kfp, MODINFOMD_KERNEND); bcopy(&kernend, md->md_data, sizeof kernend); --Apple-Mail=_9CBBB915-A277-43BE-8F77-3E9533C47C96 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii --Apple-Mail=_9CBBB915-A277-43BE-8F77-3E9533C47C96-- --Apple-Mail=_CDDAE297-F448-44F9-97F2-68B0A76CED4F Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iEYEARECAAYFAlQPJpYACgkQpgWlLWHuifafhQCfU2o4x9TmUZiKSZH95j09ITOd w/8AnRopZUhBKfUvxSvd/3rkZow6Y+pf =xqXU -----END PGP SIGNATURE----- --Apple-Mail=_CDDAE297-F448-44F9-97F2-68B0A76CED4F--